PATENT ABSTRACT
The present invention discloses a heterogeneity-projection hard-decision interpolation method for color reproduction, which utilizes a heterogeneity-projection method to determine the optimal edge direction and then utilizes a hard-decision rule to determine the optimal interpolation direction and obtain the information of the green color elements. The high-frequency information of the plane of the green color elements is incorporated into the processing of the planes of the red color elements and the blue color elements to reduce the restoration errors of the red and blue color elements. Therefore, the present invention can decrease the interpolation-direction errors and achieve a higher PSNR and a better visual effect.

PATENT DESCRIPTION
BACKGROUND OF THE INVENTION 
       [0001]    1. Field of the Invention 
         [0002]    The present invention relates to an interpolation method for color reproduction, particularly to a heterogeneity-projection hard-decision interpolation method for color reproduction. 
         [0003]    2. Description of the Related Art 
         [0004]    The digital camera uses a lens to project scenes onto a Charge Coupled Device (CCD), and the CCD transforms the scenes into digital image signals, and next, the digital image signals are processed by electronic circuits and then stored in a storage medium. The CCD cannot sense light colors but can only sense light intensity. Therefore, a color-separation filter needs to be arranged before the light-sensing element for a digital sampling. Generally, the color-separation filter uses the RGB (three primary colors) color model. The color components separately acquired by three CCD&#39;s are combined into a full-color image. In considering cost and volume, the digital camera usually uses only a single CCD. Each pixel only has the gray-level value of one of three primary colors, and the other two color elements are lost. Therefore, the result obtained by the CCD needs to be processed using an interpolation algorithm to reconstruct the missing color elements. 
         [0005]    Generally, image interpolation methods may be classified into the fixed type and the non-fixed type. In the fixed type image interpolation method, the weight values of the neighboring pixels are fixed in reproducing the missing color element. The fixed type method lacks the edge detection capability; thus, the edges of the output image appear blurred, and the detailed texture cannot be recovered well. In the non-fixed type image interpolation method, the weight values of the neighboring pixels are unfixed in reproducing the missing color element. The non-fixed type method possesses the edge detection capability; thus, the edge blurs of the output image are greatly reduced in the horizontal and vertical directions. However, the detailed texture cannot be recovered well in the non-fixed type method either. 
         [0006]    A Taiwan patent No. 00548956 proposed a “Color Interpolation Method For Digital Images”. However, such a conventional technology has color distortion in the rebuilt digital color images. Besides, the detailed texture cannot be well recovered either. 
         [0007]    Accordingly, the present invention proposes a heterogeneity-projection hard-decision interpolation method for color reproduction, which can effectively interpolate the edges of digital images and can sensitively detect the passage of an edge and can thus effectively recover the details of texture. 
       SUMMARY OF THE INVENTION 
       [0008]    The primary objective of the present invention is to provide a heterogeneity-projection hard-decision interpolation method for color reproduction, which uses a heterogeneity-projection technology and a hard-decision rule to determine the optimal interpolation direction and reduce the interpolation-direction errors occurring in interpolation stage. 
         [0009]    Another objective of the present invention is to provide a heterogeneity-projection hard-decision interpolation method for color reproduction, which can effectively reduce the edge blurs in the rebuilt digital color image and can effectively recover the details of the texture in the rebuilt digital color image. 
         [0010]    Still another objective of the present invention is to provide a heterogeneity-projection hard-decision interpolation method for color reproduction, which can reproduce the missing color elements of the pixels formed by a color filter array (CFA) and make the color performance of the rebuilt digital color image more close to the original object. 
         [0011]    Further another objective of the present invention is to provide a heterogeneity-projection hard-decision interpolation method for color reproduction, which can integrate with the existing fixed type and non-fixed type image interpolation methods and promote the performance of the existing interpolation methods. 
         [0012]    According to one aspect of the present invention, the heterogeneity-projection hard-decision interpolation method for color reproduction comprises the following steps: acquiring an image to form a Bayer-pattern digital image, which consists of a plurality of pixels, including a plurality of red pixels, blue pixels and green pixels; utilizing a heterogeneity-projection method to horizontally and vertically project the original Bayer-pattern digital image onto horizontal and vertical heterogeneity maps, and utilizing an image-restoration technology to obtain the optimized horizontal and vertical heterogeneity maps; utilizing the obtained optimized horizontal and vertical heterogeneity maps to form horizontal, vertical and smooth subsets according to a hard-decision rule; respectively performing horizontal, vertical and average interpolations inside the horizontal, vertical and smooth subsets to reproduce all the missing color elements; utilizing the current red color elements and the current blue color elements of all the pixels to correct the current green color elements of all the pixels and obtain a color-corrected plane of green color elements; utilizing the color-corrected green color elements of all the pixels to correct the current red color elements and the current blue color elements of all the pixels and obtain a color-corrected plane of red color elements and a color-corrected plane of blue color elements; and repeating the last two steps several times to obtain a color-corrected digital color image. 
         [0013]    To enable the objectives, technical contents, characteristics and accomplishments of the present invention to be easily understood, the embodiments of the present invention are to be described in detail in cooperation with the attached drawings below. 
         [0014]    The file of this Patent contains at least one Drawing Figure executed in color. Copies of the Patent with color Drawings will be provided by the Patent and Trademark Office upon request and payment of the necessary fee. 
     
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0015]      FIG. 1  is a flowchart of the method of the present invention; 
           [0016]      FIG. 2  is a diagram schematically showing a Bayer-pattern color filter array; 
           [0017]      FIG. 3  is a diagram schematically showing that a horizontal adaptive-filtering process is performed on an element H h  of the horizontal heterogeneity map H h     —     map ; 
           [0018]      FIG. 4  is a diagram schematically showing that a vertical adaptive-filtering process is performed on an element H v  of the horizontal heterogeneity map H v     —     map ; 
           [0019]      FIG. 5  is a diagram schematically showing the calculation for the missing green color elements of the red pixels according to the present invention; 
           [0020]      FIG. 6  is a diagram schematically showing the calculation for the missing green color elements of the blue pixels according to the present invention; 
           [0021]      FIG. 7  is a diagram schematically showing the calculation for the missing red color elements of the blue pixels according to the present invention; 
           [0022]      FIG. 8  is a diagram schematically showing the calculation for the missing red color elements of the green pixels according to the present invention; 
           [0023]      FIG. 9  is a diagram schematically showing the calculation for the missing blue color elements of the red pixels according to the present invention; 
           [0024]      FIG. 10  is a diagram schematically showing the calculation for the missing blue color elements of the green pixels according to the present invention; 
           [0025]      FIG. 11  is a diagram schematically showing the calculation for a new color-difference plane G r ; 
           [0026]      FIG. 12  is a diagram schematically showing the calculation for a new color-difference plane G b ; 
           [0027]      FIG. 13  is a diagram schematically showing the calculation for a new color-difference plane R g ; 
           [0028]      FIG. 14  is a diagram schematically showing the calculation for a new color-difference plane B g ; 
           [0029]      FIG. 15  is a diagram showing the flowchart of the process of from Step S 1  to Step S 4  and the results of the process; 
           [0030]      FIG. 16  is a diagram showing the flowchart of the process of from Step S 5  to Step S 7  and the results of the process; 
           [0031]      FIGS. 17(A) to 17(D)  are diagrams to compare the result of the method of the present invention with the results of other methods; and 
           [0032]      FIGS.18(A) to 18(D)  are further diagrams to compare the result of the method of the present invention with the results of other methods. 
           [0033]      FIG. 19  is the image demosaiced by Lu&#39;s method. 
           [0034]      FIG. 20  is the image demosaiced by Gunturk&#39;s method. 
           [0035]      FIG. 21  is the image demosaiced by the method of present invention. 
           [0036]      FIG. 22  is the image demosaiced by the method of present invention. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0037]    The present invention pertains to a heterogeneity-projection hard-decision interpolation method for color reproduction, which uses a heterogeneity-projection technology and a hard-decision rule to determine the optimal interpolation direction and reduce the interpolation-direction errors occurring in performing interpolations. 
         [0038]    Refer to  FIG. 1  a flowchart of the method of the present invention. The method of the present invention, which can restore the missing color elements for each pixel in a Bayer-pattern digital image, comprises the following steps: acquiring a Bayer-pattern digital image from a single CCD with CFA (Step S 1 ); obtain the optimized horizontal and vertical heterogeneity maps (Step S 2 ); forming horizontal, vertical and smooth subsets according to a hard-decision rule (Step S 3 ); restoring all the missing color elements (Step S 4 ); performing a color modification on the plane of the existing green color elements (Step S 5 ); respectively performing color modifications on the planes of the existing red and blue color elements (Step S 6 ); repeating Step S 5  and Step S 6  one to three times (Step S 7 ). The abovementioned steps will be described in detail below. 
         [0039]    In Step S 1 , a single CCD and a Bayer-pattern CFA are used to obtain a Bayer-pattern digital image, which consists of a plurality of pixels, including red, blue and green pixels. Each pixel only has the gray-level value of a single color element. Refer to  FIG. 2  a diagram schematically showing a Bayer-pattern CFA. 
         [0040]    In Step S 2 , the optimized horizontal and vertical heterogeneity maps are to be worked out. Firstly, an N×1 heterogeneity-projection vector is obtained with Equation (1): 
         [0000]        P   N×1   =H   N×M   V   M×1    (1) 
         [0000]    wherein the N×M matrix H N×M  is obtained with Equation (2): 
         [0000]        H   N×M =[1−1−1 1] T {circle around (×)}eye( M )   (2) 
         [0000]    and the M×1 vector V M×1  is obtained with Equation (3): 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       V 
                       
                         M 
                         × 
                         1 
                       
                     
                     = 
                     
                       
                         ∏ 
                         
                           i 
                           = 
                           1 
                         
                         
                           M 
                           - 
                           1 
                         
                       
                        
                       
                           
                       
                        
                       
                         
                           
                             [ 
                             
                               1 
                                
                               
                                   
                               
                               - 
                               1 
                             
                             ] 
                           
                           
                             T 
                           
                         
                         ⊗ 
                         
                           eye 
                            
                           
                             ( 
                             
                               M 
                               - 
                               i 
                             
                             ) 
                           
                         
                       
                     
                   
                   ; 
                 
               
               
                 
                   ( 
                   3 
                   ) 
                 
               
             
           
         
       
     
         [0000]    M=N−3, and N is an integer equal to or greater than 5; {circle around (×)} denotes the 2D convolution operator, and eye(M) denotes an M×M identity matrix. The heterogeneity-projection vector P N×1  is used to obtain the horizontal and vertical heterogeneity maps with Equation (4) and (5): 
         [0000]        H   h     —     map =|Bayer {circle around (×)} P   N×1   T |  (4) 
         [0000]        H   v     —     map =|Bayer {circle around (×)} P   N×1 |  (5), 
         [0000]    wherein Bayer denotes the original Bayer-pattern digital image. 
         [0041]    Image-restoration technologies, such as the mean filter, the median filter and the adaptive filter, are used to directionally filter out the noise of the horizontal and vertical heterogeneity maps obtained with Equation (4) and (5), i.e. to filter out the horizontal noise of the horizontal heterogeneity map H h     —     map  and filter out the vertical noise of the vertical heterogeneity map H v     —     map . 
         [0042]    Herein, a directional noise-filtering method implemented with the adaptive filter is to be described below. Refer to  FIG. 3  a diagram schematically showing that a horizontal-direction adaptive filtering is performed on one pixel H h  of the horizontal heterogeneity map H h     —     map . The horizontal-direction adaptive filtering is performed on each pixel H h  of the horizontal heterogeneity map H h     —     map  according to Equation (6): 
         [0000]    
       
         
           
             
               
                 
                   
                     H 
                     h 
                     * 
                   
                   = 
                   
                     
                       
                         H 
                         _ 
                       
                       h 
                       L 
                     
                     + 
                     
                       
                         
                           δ 
                            
                           
                               
                           
                            
                           
                             H 
                             h 
                             L 
                           
                         
                         
                           
                             δ 
                              
                             
                                 
                             
                              
                             
                               H 
                               h 
                               L 
                             
                           
                           + 
                           
                             δ 
                              
                             
                                 
                             
                              
                             
                               H 
                               h 
                               R 
                             
                           
                         
                       
                        
                       
                         ( 
                         
                           
                             
                               H 
                               _ 
                             
                             h 
                             R 
                           
                           - 
                           
                             
                               H 
                               _ 
                             
                             h 
                             L 
                           
                         
                         ) 
                       
                     
                   
                 
               
               
                 
                   ( 
                   6 
                   ) 
                 
               
             
           
         
       
     
         [0000]    wherein H* h  denotes the optimal estimated value of the adaptively filtered pixel H h ; H h   L  denotes the left neighboring pixel of H h , and (  H   h   L ,δH h   L ) are respectively the local mean and variance of the H h   L -centered 1×N window; H h   R  denotes the right neighboring pixel of H h , and (  H   h   R ,δH h   R ) are respectively the local mean and variance of the H h   R -centered 1×N window. After all the pixels of H   —     map  are processed with Equation (6), the optimized horizontal heterogeneity map H* h     —     map  is obtained. 
         [0043]    Refer to  FIG. 4  a diagram schematically showing that a vertical-direction adaptive filtering is performed on one pixel H v  of the vertical heterogeneity map H h     —     map . The vertical-direction adaptive filtering is performed on each pixel H v  of the vertical heterogeneity map H v     —     map  according to Equation (7): 
         [0000]    
       
         
           
             
               
                 
                   
                     H 
                     v 
                     * 
                   
                   = 
                   
                     
                       
                         H 
                         _ 
                       
                       v 
                       U 
                     
                     + 
                     
                       
                         
                           δ 
                            
                           
                               
                           
                            
                           
                             H 
                             v 
                             U 
                           
                         
                         
                           
                             δ 
                              
                             
                                 
                             
                              
                             
                               H 
                               v 
                               U 
                             
                           
                           + 
                           
                             δ 
                              
                             
                                 
                             
                              
                             
                               H 
                               v 
                               D 
                             
                           
                         
                       
                        
                       
                         ( 
                         
                           
                             
                               H 
                               _ 
                             
                             v 
                             D 
                           
                           - 
                           
                             
                               H 
                               _ 
                             
                             v 
                             U 
                           
                         
                         ) 
                       
                     
                   
                 
               
               
                 
                   ( 
                   7 
                   ) 
                 
               
             
           
         
       
     
         [0000]    wherein H* v  denotes the optimal estimated value of the adaptively filtered pixel H v ; H v   U  denotes the upside neighboring pixel of H v , and (  H   v   U ,δH v   U ) are respectively the local mean and variance of the H v   U -centered N×1 window; H v   D  denotes the downside neighboring pixel of H v , and (  H   v   D ,δH v   D ) are respectively the local mean and variance of the H v   D -centered N×1 window. 
         [0044]    In Step S 3 , the hard-decision rule is used to separate a horizontal subset Ω h , a vertical subset Ω v  and a smooth subset Ω s  from the image according to Equations (8), (9) and (10): 
         [0000]      Ω h ≡{( x,y )| H*   h     —     map ( x,y )&lt;α H*   v     —     map ( x,y )}  (8) 
         [0000]      Ω v ≡{( x,y )| H*   v     —     map ( x,y )&lt;α H*   h     —     map ( x,y )}  (9) 
         [0000]      Ω s ≡{(x,y)|(x,y)∉Ω h ,(x,y)∉Ω v }  (10) 
         [0000]    wherein (x,y) is the position of one pixel of the image, and α is a scalar factor within 0 to 1. 
         [0045]    In Step S 4 , all the missing color elements are restored. Firstly, all the missing green color elements are restored. Then, the restored green color elements are used to restore all the missing red and blue color elements. 
       Restoring the Green Color Elements 
       [0046]    Refer to  FIG. 5  a diagram schematically showing that the missing green color elements of the red pixels are worked out according to the method of the present invention. For restoring the green color elements of the red pixels, the color-adjusted values and the corresponding weight values of four green pixels surrounding each one of a plurality of the red pixels are respectively worked out according to Equation (11): 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         G 
                         ^ 
                       
                       i 
                     
                     = 
                     
                       
                         G 
                         i 
                       
                       + 
                       
                         
                           ( 
                           
                             R 
                             - 
                             
                               R 
                               i 
                             
                           
                           ) 
                         
                         / 
                         2 
                       
                     
                   
                   , 
                   
                     
 
                   
                    
                   
                     
                       w 
                       i 
                     
                     = 
                     
                       
                         e 
                         i 
                       
                       / 
                       
                         
                           ∑ 
                           
                             k 
                             = 
                             1 
                           
                           4 
                         
                          
                         
                           e 
                           i 
                         
                       
                     
                   
                   , 
                   
                     i 
                     = 
                     
                       1 
                       ~ 
                       4 
                     
                   
                 
               
               
                 
                   ( 
                   11 
                   ) 
                 
               
             
           
         
       
     
         [0000]    wherein Ĝ i  denotes the color-adjusted value of one of four green pixels surrounding one red pixel, and w i  is the weight value corresponding to the color-adjusted value, and e i  is the edge indicator of one of four green pixels surrounding the red pixel. The edge indicator e i  can be worked out with Equations (12-1) to (12-4): 
         [0000]    
       
         
           
             
               
                 
                   
                     e 
                     1 
                   
                   = 
                   
                     
                       ( 
                       
                         
                           
                             
                               1 
                               + 
                               
                                  
                                 
                                   
                                     G 
                                     1 
                                   
                                   - 
                                   
                                     G 
                                     3 
                                   
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     G 
                                     5 
                                   
                                   - 
                                   
                                     G 
                                     1 
                                   
                                 
                                  
                               
                               + 
                             
                           
                         
                         
                           
                             
                               
                                  
                                 
                                   
                                     R 
                                     1 
                                   
                                   - 
                                   R 
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     
                                       G 
                                       9 
                                     
                                     - 
                                     
                                       G 
                                       4 
                                     
                                   
                                   2 
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     
                                       G 
                                       10 
                                     
                                     - 
                                     
                                       G 
                                       2 
                                     
                                   
                                   2 
                                 
                                  
                               
                             
                           
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     12 
                      
                     
                       - 
                     
                      
                     1 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     2 
                   
                   = 
                   
                     
                       ( 
                       
                         
                           
                             
                               1 
                               + 
                               
                                  
                                 
                                   
                                     G 
                                     2 
                                   
                                   - 
                                   
                                     G 
                                     4 
                                   
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     G 
                                     6 
                                   
                                   - 
                                   
                                     G 
                                     2 
                                   
                                 
                                  
                               
                               + 
                             
                           
                         
                         
                           
                             
                               
                                  
                                 
                                   
                                     R 
                                     2 
                                   
                                   - 
                                   R 
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     
                                       G 
                                       11 
                                     
                                     - 
                                     
                                       G 
                                       1 
                                     
                                   
                                   2 
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     
                                       G 
                                       12 
                                     
                                     - 
                                     
                                       G 
                                       3 
                                     
                                   
                                   2 
                                 
                                  
                               
                             
                           
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     12 
                      
                     
                       - 
                     
                      
                     2 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     3 
                   
                   = 
                   
                     
                       ( 
                       
                         
                           
                             
                               1 
                               + 
                               
                                  
                                 
                                   
                                     G 
                                     1 
                                   
                                   - 
                                   
                                     G 
                                     3 
                                   
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     G 
                                     3 
                                   
                                   - 
                                   
                                     G 
                                     7 
                                   
                                 
                                  
                               
                               + 
                             
                           
                         
                         
                           
                             
                               
                                  
                                 
                                   R 
                                   - 
                                   
                                     R 
                                     3 
                                   
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     
                                       G 
                                       2 
                                     
                                     - 
                                     
                                       G 
                                       13 
                                     
                                   
                                   2 
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     
                                       G 
                                       4 
                                     
                                     - 
                                     
                                       G 
                                       14 
                                     
                                   
                                   2 
                                 
                                  
                               
                             
                           
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     12 
                      
                     
                       - 
                     
                      
                     3 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     4 
                   
                   = 
                   
                     
                       ( 
                       
                         
                           
                             
                               1 
                               + 
                               
                                  
                                 
                                   
                                     G 
                                     2 
                                   
                                   - 
                                   
                                     G 
                                     4 
                                   
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     G 
                                     4 
                                   
                                   - 
                                   
                                     G 
                                     8 
                                   
                                 
                                  
                               
                               + 
                             
                           
                         
                         
                           
                             
                               
                                  
                                 
                                   R 
                                   - 
                                   
                                     R 
                                     4 
                                   
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     
                                       G 
                                       1 
                                     
                                     - 
                                     
                                       G 
                                       16 
                                     
                                   
                                   2 
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     
                                       G 
                                       3 
                                     
                                     - 
                                     
                                       G 
                                       15 
                                     
                                   
                                   2 
                                 
                                  
                               
                             
                           
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     12 
                      
                     
                       - 
                     
                      
                     4 
                   
                   ) 
                 
               
             
           
         
       
     
       If the position of a red pixel (x R ,y R ) belongs to the smooth subset Ω s , the green color element of the red pixel is worked out with Equation (13): 
       [0047]    
       
         
           
             
               
                 
                   G 
                   = 
                   
                     
                       ∑ 
                       
                         i 
                         = 
                         1 
                       
                       4 
                     
                      
                     
                       
                         w 
                         i 
                       
                        
                       
                         
                           G 
                           ^ 
                         
                         i 
                       
                     
                   
                 
               
               
                 
                   ( 
                   13 
                   ) 
                 
               
             
           
         
       
     
       If the position of a red pixel (x R ,y R ) belongs to the horizontal subset Ω h , the green color element of the red pixel is worked out with Equation (14): 
       [0048]    
       
         
           
             
               
                 
                   G 
                   = 
                   
                     
                       
                         
                           w 
                           2 
                         
                          
                         
                           
                             G 
                             ^ 
                           
                           2 
                         
                       
                       + 
                       
                         
                           w 
                           4 
                         
                          
                         
                           
                             G 
                             ^ 
                           
                           4 
                         
                       
                     
                     
                       
                         w 
                         2 
                       
                       + 
                       
                         w 
                         4 
                       
                     
                   
                 
               
               
                 
                   ( 
                   14 
                   ) 
                 
               
             
           
         
       
     
       If the position of a red pixel (x R ,y R ) belongs to the vertical subset Ω v , the green color element of the red pixel is worked out with Equation (15): 
       [0049]    
       
         
           
             
               
                 
                   G 
                   = 
                   
                     
                       
                         
                           w 
                           1 
                         
                          
                         
                           
                             G 
                             ^ 
                           
                           1 
                         
                       
                       + 
                       
                         
                           w 
                           3 
                         
                          
                         
                           
                             G 
                             ^ 
                           
                           3 
                         
                       
                     
                     
                       
                         w 
                         1 
                       
                       + 
                       
                         w 
                         3 
                       
                     
                   
                 
               
               
                 
                   ( 
                   15 
                   ) 
                 
               
             
           
         
       
     
         [0050]    Refer to  FIG. 6  a diagram schematically showing that the missing green color elements of the blue pixels are worked out according to the method of the present invention. For restoring the green color elements of the blue pixels, the color-adjusted values and the corresponding weight values of four green pixels surrounding each one of a plurality of the blue pixels are respectively worked out according to Equation (16): 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         G 
                         ^ 
                       
                       i 
                     
                     = 
                     
                       
                         G 
                         i 
                       
                       + 
                       
                         
                           ( 
                           
                             B 
                             - 
                             
                               B 
                               i 
                             
                           
                           ) 
                         
                         / 
                         2 
                       
                     
                   
                   , 
                   
                     
 
                   
                    
                   
                     
                       w 
                       i 
                     
                     = 
                     
                       
                         e 
                         i 
                       
                       / 
                       
                         
                           ∑ 
                           
                             k 
                             = 
                             1 
                           
                           4 
                         
                          
                         
                             
                         
                          
                         
                           e 
                           k 
                         
                       
                     
                   
                   , 
                   
                     
 
                   
                    
                   
                     i 
                     = 
                     
                       1 
                       ~ 
                       4 
                     
                   
                 
               
               
                 
                   ( 
                   16 
                   ) 
                 
               
             
           
         
       
     
         [0000]    wherein Ĝ i  denotes the color-adjusted value of one of four green pixels surrounding one blue pixel, and w i  is the weight value corresponding to the color-adjusted value, and e i  is the edge indicator of one of four green pixels surrounding the blue pixel. The edge indicator e i  can be worked out with Equations (17-1) to (17-4): 
         [0000]    
       
         
           
             
               
                 
                   
                     e 
                     1 
                   
                   = 
                   
                     
                       ( 
                       
                         
                           
                             
                               1 
                               + 
                               
                                  
                                 
                                   
                                     G 
                                     1 
                                   
                                   - 
                                   
                                     G 
                                     3 
                                   
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     G 
                                     5 
                                   
                                   - 
                                   
                                     G 
                                     1 
                                   
                                 
                                  
                               
                               + 
                             
                           
                         
                         
                           
                             
                               
                                  
                                 
                                   
                                     B 
                                     1 
                                   
                                   - 
                                   B 
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     
                                       G 
                                       9 
                                     
                                     - 
                                     
                                       G 
                                       4 
                                     
                                   
                                   2 
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     
                                       G 
                                       10 
                                     
                                     - 
                                     
                                       G 
                                       2 
                                     
                                   
                                   2 
                                 
                                  
                               
                             
                           
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     17 
                      
                     
                       - 
                     
                      
                     1 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     2 
                   
                   = 
                   
                     
                       ( 
                       
                         
                           
                             
                               1 
                               + 
                               
                                  
                                 
                                   
                                     G 
                                     2 
                                   
                                   - 
                                   
                                     G 
                                     4 
                                   
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     G 
                                     6 
                                   
                                   - 
                                   
                                     G 
                                     2 
                                   
                                 
                                  
                               
                               + 
                             
                           
                         
                         
                           
                             
                               
                                  
                                 
                                   
                                     B 
                                     2 
                                   
                                   - 
                                   B 
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     
                                       G 
                                       11 
                                     
                                     - 
                                     
                                       G 
                                       1 
                                     
                                   
                                   2 
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     
                                       G 
                                       12 
                                     
                                     - 
                                     
                                       G 
                                       3 
                                     
                                   
                                   2 
                                 
                                  
                               
                             
                           
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     17 
                      
                     
                       - 
                     
                      
                     2 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     3 
                   
                   = 
                   
                     
                       ( 
                       
                         
                           
                             
                               1 
                               + 
                               
                                  
                                 
                                   
                                     G 
                                     1 
                                   
                                   - 
                                   
                                     G 
                                     3 
                                   
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     G 
                                     3 
                                   
                                   - 
                                   
                                     G 
                                     7 
                                   
                                 
                                  
                               
                               + 
                             
                           
                         
                         
                           
                             
                               
                                  
                                 
                                   B 
                                   - 
                                   
                                     B 
                                     3 
                                   
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     
                                       G 
                                       2 
                                     
                                     - 
                                     
                                       G 
                                       13 
                                     
                                   
                                   2 
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     
                                       G 
                                       4 
                                     
                                     - 
                                     
                                       G 
                                       14 
                                     
                                   
                                   2 
                                 
                                  
                               
                             
                           
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     17 
                      
                     
                       - 
                     
                      
                     3 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     4 
                   
                   = 
                   
                     
                       ( 
                       
                         
                           
                             
                               1 
                               + 
                               
                                  
                                 
                                   
                                     G 
                                     2 
                                   
                                   - 
                                   
                                     G 
                                     4 
                                   
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     G 
                                     4 
                                   
                                   - 
                                   
                                     G 
                                     8 
                                   
                                 
                                  
                               
                               + 
                             
                           
                         
                         
                           
                             
                               
                                  
                                 
                                   B 
                                   - 
                                   
                                     B 
                                     4 
                                   
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     
                                       G 
                                       1 
                                     
                                     - 
                                     
                                       G 
                                       16 
                                     
                                   
                                   2 
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     
                                       G 
                                       3 
                                     
                                     - 
                                     
                                       G 
                                       15 
                                     
                                   
                                   2 
                                 
                                  
                               
                             
                           
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     17 
                      
                     
                       - 
                     
                      
                     4 
                   
                   ) 
                 
               
             
           
         
       
     
       The missing green color element of one blue pixel is worked out with one of Equations (13) to (15). 
     Restoring the Red Color Elements 
       [0051]    Refer to  FIG. 7  a diagram schematically showing that the missing red color elements of the blue pixels are worked out according to the method of the present invention. For restoring the red color elements of the blue pixels, the bilinear interpolation method is used to preliminarily restore all the missing red color elements, and the plane of the restored green color elements is subtracted from the plane of the preliminarily restored red color elements to obtain a color-difference plane R g (R g =R−G) therebetween. The missing red color element of one blue pixel is worked out with Equation (18): 
         [0000]    
       
         
           
             
               
                 
                   
                     R 
                     g 
                   
                   = 
                   
                     
                       
                         
                           e 
                           
                             a 
                              
                             
                                 
                             
                              
                             1 
                           
                         
                          
                         
                           
                             R 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             1 
                           
                         
                       
                       + 
                       
                         
                           e 
                           
                             a 
                              
                             
                                 
                             
                              
                             2 
                           
                         
                          
                         
                           
                             R 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             2 
                           
                         
                       
                       + 
                       
                         
                           e 
                           
                             a 
                              
                             
                                 
                             
                              
                             3 
                           
                         
                          
                         
                           
                             R 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             3 
                           
                         
                       
                       + 
                       
                         
                           e 
                           
                             a 
                              
                             
                                 
                             
                              
                             4 
                           
                         
                          
                         
                           
                             R 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             4 
                           
                         
                       
                     
                     
                       
                         e 
                         
                           a 
                            
                           
                               
                           
                            
                           1 
                         
                       
                       + 
                       
                         e 
                         
                           a 
                            
                           
                               
                           
                            
                           2 
                         
                       
                       + 
                       
                         e 
                         
                           a 
                            
                           
                               
                           
                            
                           3 
                         
                       
                       + 
                       
                         e 
                         
                           a 
                            
                           
                               
                           
                            
                           4 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   18 
                   ) 
                 
               
             
           
         
       
     
         [0000]    wherein {circumflex over (R)} gi  i=1˜4 denotes the adjusted value of red color difference, and e ai  i=1˜4 denotes the edge indicator of color difference. 
         [0052]    Refer to  FIG. 8  a diagram schematically showing that the missing red color elements of the green pixels are worked out according to the method of the present invention. The restored green color elements and the restored red color elements of the blue pixels are used to restore the missing red color elements of the green pixels. If the position of a green pixel (x G ,y G ) belongs to the smooth subset Ω s , the missing red color element of the green pixel is worked out with Equation (19): 
         [0000]    
       
         
           
             
               
                 
                   
                     R 
                     g 
                   
                   = 
                   
                     
                       
                         
                           e 
                           
                             b 
                              
                             
                                 
                             
                              
                             1 
                           
                         
                          
                         
                           
                             R 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             1 
                           
                         
                       
                       + 
                       
                         
                           e 
                           
                             b 
                              
                             
                                 
                             
                              
                             2 
                           
                         
                          
                         
                           
                             R 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             2 
                           
                         
                       
                       + 
                       
                         
                           e 
                           
                             b 
                              
                             
                                 
                             
                              
                             3 
                           
                         
                          
                         
                           
                             R 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             3 
                           
                         
                       
                       + 
                       
                         
                           e 
                           
                             b 
                              
                             
                                 
                             
                              
                             4 
                           
                         
                          
                         
                           
                             R 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             4 
                           
                         
                       
                     
                     
                       
                         e 
                         
                           b 
                            
                           
                               
                           
                            
                           1 
                         
                       
                       + 
                       
                         e 
                         
                           b 
                            
                           
                               
                           
                            
                           2 
                         
                       
                       + 
                       
                         e 
                         
                           b 
                            
                           
                               
                           
                            
                           3 
                         
                       
                       + 
                       
                         e 
                         
                           b 
                            
                           
                               
                           
                            
                           4 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   19 
                   ) 
                 
               
             
           
         
       
     
       If the position of a green pixel (x G ,y G ) belongs to the horizontal subset Ω h , the missing red color element of the green pixel is worked out with Equation (20): 
       [0053]    
       
         
           
             
               
                 
                   
                     R 
                     g 
                   
                   = 
                   
                     
                       
                         
                           e 
                           
                             b 
                              
                             
                                 
                             
                              
                             2 
                           
                         
                          
                         
                           
                             R 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             2 
                           
                         
                       
                       + 
                       
                         
                           e 
                           
                             b 
                              
                             
                                 
                             
                              
                             4 
                           
                         
                          
                         
                           
                             R 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             4 
                           
                         
                       
                     
                     
                       
                         e 
                         
                           b 
                            
                           
                               
                           
                            
                           2 
                         
                       
                       + 
                       
                         e 
                         
                           b 
                            
                           
                               
                           
                            
                           4 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   20 
                   ) 
                 
               
             
           
         
       
     
       If the position of a green pixel (x G ,y G ) belongs to the vertical subset Ω v , the missing red color element of the green pixel is worked out with Equation (21): 
       [0054]    
       
         
           
             
               
                 
                   
                     R 
                     g 
                   
                   = 
                   
                     
                       
                         
                           e 
                           
                             b 
                              
                             
                                 
                             
                              
                             1 
                           
                         
                          
                         
                           
                             R 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             1 
                           
                         
                       
                       + 
                       
                         
                           e 
                           
                             b 
                              
                             
                                 
                             
                              
                             3 
                           
                         
                          
                         
                           
                             R 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             3 
                           
                         
                       
                     
                     
                       
                         e 
                         
                           b 
                            
                           
                               
                           
                            
                           1 
                         
                       
                       + 
                       
                         e 
                         
                           b 
                            
                           
                               
                           
                            
                           3 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   21 
                   ) 
                 
               
             
           
         
       
     
       The adjusted values of red color difference {circumflex over (R)} gi  i=1˜4 in Equations (18) to (21) can be worked out with Equations (22-1) to (22-4): 
       [0055]    
       
         
           
             
               
                 
                   
                     
                       R 
                       ^ 
                     
                     
                       g 
                        
                       
                           
                       
                        
                       1 
                     
                   
                   = 
                   
                     
                       R 
                       
                         g 
                          
                         
                             
                         
                          
                         1 
                       
                     
                     + 
                     
                       
                         
                           R 
                           
                             g 
                              
                             
                                 
                             
                              
                             3 
                           
                         
                         - 
                         
                           R 
                           
                             g 
                              
                             
                                 
                             
                              
                             1 
                           
                         
                       
                       2 
                     
                   
                 
               
               
                 
                   ( 
                   
                     22 
                      
                     
                       - 
                     
                      
                     1 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     
                       R 
                       ^ 
                     
                     
                       g 
                        
                       
                           
                       
                        
                       2 
                     
                   
                   = 
                   
                     
                       R 
                       
                         g 
                          
                         
                             
                         
                          
                         2 
                       
                     
                     + 
                     
                       
                         
                           R 
                           
                             g 
                              
                             
                                 
                             
                              
                             4 
                           
                         
                         - 
                         
                           R 
                           
                             g 
                              
                             
                                 
                             
                              
                             2 
                           
                         
                       
                       2 
                     
                   
                 
               
               
                 
                   ( 
                   
                     22 
                      
                     
                       - 
                     
                      
                     2 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     
                       R 
                       ^ 
                     
                     
                       g 
                        
                       
                           
                       
                        
                       3 
                     
                   
                   = 
                   
                     
                       R 
                       
                         g 
                          
                         
                             
                         
                          
                         3 
                       
                     
                     + 
                     
                       
                         
                           R 
                           
                             g 
                              
                             
                                 
                             
                              
                             1 
                           
                         
                         - 
                         
                           R 
                           
                             g 
                              
                             
                                 
                             
                              
                             3 
                           
                         
                       
                       2 
                     
                   
                 
               
               
                 
                   ( 
                   
                     22 
                      
                     
                       - 
                     
                      
                     3 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     
                       R 
                       ^ 
                     
                     
                       g 
                        
                       
                           
                       
                        
                       4 
                     
                   
                   = 
                   
                     
                       R 
                       
                         g 
                          
                         
                             
                         
                          
                         4 
                       
                     
                     + 
                     
                       
                         
                           R 
                           
                             g 
                              
                             
                                 
                             
                              
                             2 
                           
                         
                         - 
                         
                           R 
                           
                             g 
                              
                             
                                 
                             
                              
                             4 
                           
                         
                       
                       2 
                     
                   
                 
               
               
                 
                   ( 
                   
                     22 
                      
                     
                       - 
                     
                      
                     4 
                   
                   ) 
                 
               
             
           
         
       
     
         [0000]    Refer to  FIG. 7  again. The edge indicators of color difference e ai  i=1˜4 can be obtained with Equations (23-1) to (23-4): 
         [0000]    
       
         
           
             
               
                 
                   
                     e 
                     
                       a 
                        
                       
                           
                       
                        
                       1 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                               - 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   5 
                                 
                               
                               - 
                               
                                 2 
                                  
                                 
                                   R 
                                   
                                     g 
                                      
                                     
                                         
                                     
                                      
                                     1 
                                   
                                 
                               
                               + 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     23 
                      
                     
                       - 
                     
                      
                     1 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       a 
                        
                       
                           
                       
                        
                       2 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   4 
                                 
                               
                               - 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   4 
                                 
                               
                               - 
                               
                                 2 
                                  
                                 
                                   R 
                                   
                                     g 
                                      
                                     
                                         
                                     
                                      
                                     2 
                                   
                                 
                               
                               + 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   6 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     23 
                      
                     
                       - 
                     
                      
                     2 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       a 
                        
                       
                           
                       
                        
                       3 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                               - 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                               - 
                               
                                 2 
                                  
                                 
                                   R 
                                   
                                     g 
                                      
                                     
                                         
                                     
                                      
                                     3 
                                   
                                 
                               
                               + 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   7 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     23 
                      
                     
                       - 
                     
                      
                     3 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       a 
                        
                       
                           
                       
                        
                       4 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   4 
                                 
                               
                               - 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   8 
                                 
                               
                               - 
                               
                                 2 
                                  
                                 
                                   R 
                                   
                                     g 
                                      
                                     
                                         
                                     
                                      
                                     4 
                                   
                                 
                               
                               + 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     23 
                      
                     
                       - 
                     
                      
                     4 
                   
                   ) 
                 
               
             
           
         
       
     
         [0000]    Refer to  FIG. 8  again. The edge indicators of color difference e bi  i=1˜4 can be obtained with Equations (24-1) to (24-4): 
         [0000]    
       
         
           
             
               
                 
                   
                     e 
                     
                       b 
                        
                       
                           
                       
                        
                       1 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                               - 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                               - 
                               
                                 2 
                                  
                                 
                                   R 
                                   
                                     g 
                                      
                                     
                                         
                                     
                                      
                                     1 
                                   
                                 
                               
                               + 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   5 
                                 
                               
                             
                             2 
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     24 
                      
                     
                       - 
                     
                      
                     1 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       b 
                        
                       
                           
                       
                        
                       2 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                               - 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   4 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   6 
                                 
                               
                               - 
                               
                                 2 
                                  
                                 
                                   R 
                                   
                                     g 
                                      
                                     
                                         
                                     
                                      
                                     2 
                                   
                                 
                               
                               + 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   4 
                                 
                               
                             
                             2 
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     24 
                      
                     
                       - 
                     
                      
                     2 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       b 
                        
                       
                           
                       
                        
                       3 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                               - 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   7 
                                 
                               
                               - 
                               
                                 2 
                                  
                                 
                                   R 
                                   
                                     g 
                                      
                                     
                                         
                                     
                                      
                                     3 
                                   
                                 
                               
                               + 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                             
                             2 
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     24 
                      
                     
                       - 
                     
                      
                     3 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       b 
                        
                       
                           
                       
                        
                       4 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                               - 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   4 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                               - 
                               
                                 2 
                                  
                                 
                                   R 
                                   
                                     g 
                                      
                                     
                                         
                                     
                                      
                                     4 
                                   
                                 
                               
                               + 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   8 
                                 
                               
                             
                             2 
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     24 
                      
                     
                       - 
                     
                      
                     4 
                   
                   ) 
                 
               
             
           
         
       
     
       Then, the plane of the restored green color elements G is added to the plane of red color difference R g  to obtain all the missing red color elements (R=G+R g ). 
     Restoring the Blue Color Elements 
       [0056]    Refer to  FIG. 9  a diagram schematically showing that the missing blue color elements of the red pixels are worked out according to the method of the present invention. For restoring the blue color elements of the red pixels, the bilinear interpolation method is used to preliminarily restore all the missing red color elements, and the plane of the restored green color elements is subtracted from the plane of the preliminarily restored red color elements to obtain a color-difference plane B g  (B g   32  B−G) therebetween. The missing blue color element of a red pixel is worked out with Equation (25): 
         [0000]    
       
         
           
             
               
                 
                   
                     B 
                     g 
                   
                   = 
                   
                     
                       
                         
                           e 
                           
                             a 
                              
                             
                                 
                             
                              
                             1 
                           
                         
                          
                         
                           
                             B 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             1 
                           
                         
                       
                       + 
                       
                         
                           e 
                           
                             a 
                              
                             
                                 
                             
                              
                             2 
                           
                         
                          
                         
                           
                             B 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             2 
                           
                         
                       
                       + 
                       
                         
                           e 
                           
                             a 
                              
                             
                                 
                             
                              
                             3 
                           
                         
                          
                         
                           
                             B 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             3 
                           
                         
                       
                       + 
                       
                         
                           e 
                           
                             a 
                              
                             
                                 
                             
                              
                             4 
                           
                         
                          
                         
                           
                             B 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             4 
                           
                         
                       
                     
                     
                       
                         e 
                         
                           a 
                            
                           
                               
                           
                            
                           1 
                         
                       
                       + 
                       
                         e 
                         
                           a 
                            
                           
                               
                           
                            
                           2 
                         
                       
                       + 
                       
                         e 
                         
                           a 
                            
                           
                               
                           
                            
                           3 
                         
                       
                       + 
                       
                         e 
                         
                           a 
                            
                           
                               
                           
                            
                           4 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   25 
                   ) 
                 
               
             
           
         
       
     
         [0000]    wherein {circumflex over (B)} gi  i=1˜4 denotes the adjusted value of blue color difference, and e ai  i=1˜4 denotes the edge indicator of color difference. 
         [0057]    Refer to  FIG. 10  a diagram schematically showing that the missing blue color elements of the green pixels are worked out according to the method of the present invention. The restored green color elements and the restored blue color elements of the red pixels are used to restore the missing blue color elements of the green pixels. If the position of a green pixel (x G ,y G ) belongs to the smooth subset Ω s , the missing blue color element of the green pixel is worked out with Equation (26): 
         [0000]    
       
         
           
             
               
                 
                   
                     B 
                     g 
                   
                   = 
                   
                     
                       
                         
                           e 
                           
                             b 
                              
                             
                                 
                             
                              
                             1 
                           
                         
                          
                         
                           
                             B 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             1 
                           
                         
                       
                       + 
                       
                         
                           e 
                           
                             b 
                              
                             
                                 
                             
                              
                             2 
                           
                         
                          
                         
                           
                             B 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             2 
                           
                         
                       
                       + 
                       
                         
                           e 
                           
                             b 
                              
                             
                                 
                             
                              
                             3 
                           
                         
                          
                         
                           
                             B 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             3 
                           
                         
                       
                       + 
                       
                         
                           e 
                           
                             b 
                              
                             
                                 
                             
                              
                             4 
                           
                         
                          
                         
                           
                             B 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             4 
                           
                         
                       
                     
                     
                       
                         e 
                         
                           b 
                            
                           
                               
                           
                            
                           1 
                         
                       
                       + 
                       
                         e 
                         
                           b 
                            
                           
                               
                           
                            
                           2 
                         
                       
                       + 
                       
                         e 
                         
                           b 
                            
                           
                               
                           
                            
                           3 
                         
                       
                       + 
                       
                         e 
                         
                           b 
                            
                           
                               
                           
                            
                           4 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   26 
                   ) 
                 
               
             
           
         
       
     
       If the position of a green pixel (x G ,y G ) belongs to the horizontal subset Ω h , the missing blue color element of the green pixel is worked out with Equation (27): 
       [0058]    
       
         
           
             
               
                 
                   
                     B 
                     g 
                   
                   = 
                   
                     
                       
                         
                           e 
                           
                             b 
                              
                             
                                 
                             
                              
                             2 
                           
                         
                          
                         
                           
                             B 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             2 
                           
                         
                       
                       + 
                       
                         
                           e 
                           
                             b 
                              
                             
                                 
                             
                              
                             4 
                           
                         
                          
                         
                           
                             B 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             4 
                           
                         
                       
                     
                     
                       
                         e 
                         
                           b 
                            
                           
                               
                           
                            
                           2 
                         
                       
                       + 
                       
                         e 
                         
                           b 
                            
                           
                               
                           
                            
                           4 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   27 
                   ) 
                 
               
             
           
         
       
     
       If the position of a green pixel (x G ,y G ) belongs to the vertical subset Ω v , the missing blue color element of the green pixel is worked out with Equation (28): 
       [0059]    
       
         
           
             
               
                 
                   
                     B 
                     g 
                   
                   = 
                   
                     
                       
                         
                           e 
                           
                             b 
                              
                             
                                 
                             
                              
                             1 
                           
                         
                          
                         
                           
                             B 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             1 
                           
                         
                       
                       + 
                       
                         
                           e 
                           
                             b 
                              
                             
                                 
                             
                              
                             3 
                           
                         
                          
                         
                           
                             B 
                             ^ 
                           
                           
                             g 
                              
                             
                                 
                             
                              
                             3 
                           
                         
                       
                     
                     
                       
                         e 
                         
                           b 
                            
                           
                               
                           
                            
                           1 
                         
                       
                       + 
                       
                         e 
                         
                           b 
                            
                           
                               
                           
                            
                           3 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   28 
                   ) 
                 
               
             
           
         
       
     
       The adjusted values of blue color difference {circumflex over (B)} gi  i=1˜4 in Equations (25) to (28) can be worked out with Equations (29-1) to (29-4): 
       [0060]    
       
         
           
             
               
                 
                   
                     
                       B 
                       ^ 
                     
                     
                       g 
                        
                       
                           
                       
                        
                       1 
                     
                   
                   = 
                   
                     
                       B 
                       
                         g 
                          
                         
                             
                         
                          
                         1 
                       
                     
                     + 
                     
                       
                         
                           B 
                           
                             g 
                              
                             
                                 
                             
                              
                             3 
                           
                         
                         - 
                         
                           B 
                           
                             g 
                              
                             
                                 
                             
                              
                             1 
                           
                         
                       
                       2 
                     
                   
                 
               
               
                 
                   ( 
                   
                     29 
                      
                     
                       - 
                     
                      
                     1 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     
                       B 
                       ^ 
                     
                     
                       g 
                        
                       
                           
                       
                        
                       2 
                     
                   
                   = 
                   
                     
                       B 
                       
                         g 
                          
                         
                             
                         
                          
                         2 
                       
                     
                     + 
                     
                       
                         
                           B 
                           
                             g 
                              
                             
                                 
                             
                              
                             4 
                           
                         
                         - 
                         
                           B 
                           
                             g 
                              
                             
                                 
                             
                              
                             2 
                           
                         
                       
                       2 
                     
                   
                 
               
               
                 
                   ( 
                   
                     29 
                      
                     
                       - 
                     
                      
                     2 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     
                       B 
                       ^ 
                     
                     
                       g 
                        
                       
                           
                       
                        
                       3 
                     
                   
                   = 
                   
                     
                       B 
                       
                         g 
                          
                         
                             
                         
                          
                         3 
                       
                     
                     + 
                     
                       
                         
                           B 
                           
                             g 
                              
                             
                                 
                             
                              
                             1 
                           
                         
                         - 
                         
                           B 
                           
                             g 
                              
                             
                                 
                             
                              
                             3 
                           
                         
                       
                       2 
                     
                   
                 
               
               
                 
                   ( 
                   
                     29 
                      
                     
                       - 
                     
                      
                     3 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     
                       B 
                       ^ 
                     
                     
                       g 
                        
                       
                           
                       
                        
                       4 
                     
                   
                   = 
                   
                     
                       B 
                       
                         g 
                          
                         
                             
                         
                          
                         4 
                       
                     
                     + 
                     
                       
                         
                           B 
                           
                             g 
                              
                             
                                 
                             
                              
                             2 
                           
                         
                         - 
                         
                           B 
                           
                             g 
                              
                             
                                 
                             
                              
                             4 
                           
                         
                       
                       2 
                     
                   
                 
               
               
                 
                   ( 
                   
                     29 
                      
                     
                       - 
                     
                      
                     4 
                   
                   ) 
                 
               
             
           
         
       
     
         [0000]    Refer to  FIG. 9  again. The edge indicators of color difference e ai  i=1˜4 can be obtained with Equations (30-1) to (30-4): 
         [0000]    
       
         
           
             
               
                 
                   
                     e 
                     
                       a 
                        
                       
                           
                       
                        
                       1 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                               - 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   5 
                                 
                               
                               - 
                               
                                 2 
                                  
                                 
                                   R 
                                   
                                     g 
                                      
                                     
                                         
                                     
                                      
                                     1 
                                   
                                 
                               
                               + 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     30 
                      
                     
                       - 
                     
                      
                     1 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       a 
                        
                       
                           
                       
                        
                       2 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   4 
                                 
                               
                               - 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   4 
                                 
                               
                               - 
                               
                                 2 
                                  
                                 
                                   B 
                                   
                                     g 
                                      
                                     
                                         
                                     
                                      
                                     2 
                                   
                                 
                               
                               + 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   6 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     30 
                      
                     
                       - 
                     
                      
                     2 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       a 
                        
                       
                           
                       
                        
                       3 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                               - 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                               - 
                               
                                 2 
                                  
                                 
                                   B 
                                   
                                     g 
                                      
                                     
                                         
                                     
                                      
                                     3 
                                   
                                 
                               
                               + 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   7 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     30 
                      
                     
                       - 
                     
                      
                     3 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       a 
                        
                       
                           
                       
                        
                       4 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   4 
                                 
                               
                               - 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   8 
                                 
                               
                               - 
                               
                                 2 
                                  
                                 
                                   B 
                                   
                                     g 
                                      
                                     
                                         
                                     
                                      
                                     4 
                                   
                                 
                               
                               + 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     30 
                      
                     
                       - 
                     
                      
                     4 
                   
                   ) 
                 
               
             
           
         
       
     
         [0000]    Refer to  FIG. 10  again. The edge indicators of color difference e bi  i=1˜4 can be obtained with Equations (31-1) to (31-4): 
         [0000]    
       
         
           
             
               
                 
                   
                     e 
                     
                       b 
                        
                       
                           
                       
                        
                       1 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                               - 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                               - 
                               
                                 2 
                                  
                                 
                                   B 
                                   
                                     g 
                                      
                                     
                                         
                                     
                                      
                                     1 
                                   
                                 
                               
                               + 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   5 
                                 
                               
                             
                             2 
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     31 
                      
                     
                       - 
                     
                      
                     1 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       b 
                        
                       
                           
                       
                        
                       2 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                               - 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   4 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   6 
                                 
                               
                               - 
                               
                                 2 
                                  
                                 
                                   B 
                                   
                                     g 
                                      
                                     
                                         
                                     
                                      
                                     2 
                                   
                                 
                               
                               + 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   4 
                                 
                               
                             
                             2 
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     31 
                      
                     
                       - 
                     
                      
                     2 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       b 
                        
                       
                           
                       
                        
                       3 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                               - 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   7 
                                 
                               
                               - 
                               
                                 2 
                                  
                                 
                                   B 
                                   
                                     g 
                                      
                                     
                                         
                                     
                                      
                                     3 
                                   
                                 
                               
                               + 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                             
                             2 
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     31 
                      
                     
                       - 
                     
                      
                     3 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       b 
                        
                       
                           
                       
                        
                       4 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                               - 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   4 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                               - 
                               
                                 2 
                                  
                                 
                                   B 
                                   
                                     g 
                                      
                                     
                                         
                                     
                                      
                                     4 
                                   
                                 
                               
                               + 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   8 
                                 
                               
                             
                             2 
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     31 
                      
                     
                       - 
                     
                      
                     4 
                   
                   ) 
                 
               
             
           
         
       
     
       Then, the plane of the restored green color elements G is added to the plane of blue color difference B g  to obtain all the missing blue color elements (B=G+B g ). 
       [0061]    In Step S 5 , the plane of the obtained red color elements and the plane of the obtained blue color elements are used to correct the existing plane of the green color elements. The color-difference plane the between the planes of the green color elements and the red color elements and the color-difference plane between the planes of the green color elements and the blue color elements can be obtained with Equation (32): 
         [0000]        G   r   =G−R, G   b   =G−B    (32) 
         [0000]    Refer to  FIG. 11  a diagram schematically showing that a new color-difference plane G r  is worked out according to the method of the present invention. The new color-difference plane G r  can be obtained with Equation (33): 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       G 
                       r 
                     
                     = 
                     
                       
                         ∑ 
                         
                           j 
                           = 
                           1 
                         
                         8 
                       
                        
                       
                           
                       
                        
                       
                         
                           w 
                           j 
                         
                          
                         
                           G 
                           rj 
                         
                       
                     
                   
                   , 
                   
                     
 
                   
                    
                   
                     
                       w 
                       j 
                     
                     = 
                     
                       
                         e 
                         cj 
                       
                       
                         
                           ∑ 
                           
                             k 
                             = 
                             1 
                           
                           8 
                         
                          
                         
                             
                         
                          
                         
                           e 
                           ck 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   33 
                   ) 
                 
               
             
           
         
       
     
         [0000]    wherein e cj  j=1˜8 is the edge indicator of color difference. The edge indicators of color difference e cj  j=1˜8 can be obtained with Equations (34-1) to (34-8): 
         [0000]    
       
         
           
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       1 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   5 
                                 
                               
                               - 
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 r 
                               
                               - 
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   9 
                                 
                               
                             
                             2 
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     34 
                      
                     
                       - 
                     
                      
                     1 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       2 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                               - 
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   6 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   10 
                                 
                               
                               - 
                               
                                 G 
                                 r 
                               
                             
                             
                               2 
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     34 
                      
                     
                       - 
                     
                      
                     2 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       3 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                               - 
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   7 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   11 
                                 
                               
                               - 
                               
                                 G 
                                 r 
                               
                             
                             2 
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     34 
                      
                     
                       - 
                     
                      
                     3 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       4 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   8 
                                 
                               
                               - 
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   4 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                 
                               
                               - 
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   12 
                                 
                               
                             
                             
                               2 
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     34 
                      
                     
                       - 
                     
                      
                     4 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       5 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   5 
                                 
                               
                               - 
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   13 
                                 
                               
                               - 
                               
                                 G 
                                 r 
                               
                             
                             2 
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     34 
                      
                     
                       - 
                     
                      
                     5 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       6 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                               - 
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   6 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 r 
                               
                               - 
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   14 
                                 
                               
                             
                             
                               2 
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     34 
                      
                     
                       - 
                     
                      
                     6 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       7 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                               - 
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   7 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 r 
                               
                               - 
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   15 
                                 
                               
                             
                             
                               2 
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     34 
                      
                     
                       - 
                     
                      
                     7 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       8 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   8 
                                 
                               
                               - 
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   4 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   16 
                                 
                               
                               - 
                               
                                 G 
                                 r 
                               
                             
                             
                               2 
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     34 
                      
                     
                       - 
                     
                      
                     8 
                   
                   ) 
                 
               
             
           
         
       
     
         [0000]    Refer to  FIG. 12  a diagram schematically showing that a new color-difference plane G b  is worked out according to the method of the present invention. The new color-difference plane G b  can be obtained with Equation (35): 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       G 
                       b 
                     
                     = 
                     
                       
                         ∑ 
                         
                           j 
                           = 
                           1 
                         
                         8 
                       
                        
                       
                           
                       
                        
                       
                         
                           w 
                           j 
                         
                          
                         
                           
                             G 
                             bj 
                           
                           . 
                         
                       
                     
                   
                   , 
                   
                     
 
                   
                    
                   
                     
                       w 
                       j 
                     
                     = 
                     
                       
                         e 
                         cj 
                       
                       
                         
                           ∑ 
                           
                             k 
                             = 
                             1 
                           
                           8 
                         
                          
                         
                             
                         
                          
                         
                           e 
                           ck 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   35 
                   ) 
                 
               
             
           
         
       
     
         [0000]    wherein e cj  j=1˜8 is the edge indicator of color difference. The edge indicators of color difference e cj  j=1˜8 can be obtained with Equations (36-1) to (36-8): 
         [0000]    
       
         
           
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       1 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   5 
                                 
                               
                               - 
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 b 
                               
                               - 
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   9 
                                 
                               
                             
                             2 
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     36 
                      
                     
                       - 
                     
                      
                     1 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       2 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                               - 
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   6 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   10 
                                 
                               
                               - 
                               
                                 G 
                                 b 
                               
                             
                             
                               2 
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     36 
                      
                     
                       - 
                     
                      
                     2 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       3 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                               - 
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   7 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   11 
                                 
                               
                               - 
                               
                                 G 
                                 b 
                               
                             
                             2 
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     36 
                      
                     
                       - 
                     
                      
                     3 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       4 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   8 
                                 
                               
                               - 
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   4 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                 
                               
                               - 
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   12 
                                 
                               
                             
                             
                               2 
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     36 
                      
                     
                       - 
                     
                      
                     4 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       5 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   5 
                                 
                               
                               - 
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   13 
                                 
                               
                               - 
                               
                                 G 
                                 b 
                               
                             
                             2 
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     36 
                      
                     
                       - 
                     
                      
                     5 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       6 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                               - 
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   6 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 b 
                               
                               - 
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   14 
                                 
                               
                             
                             
                               2 
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     36 
                      
                     
                       - 
                     
                      
                     6 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       7 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                               - 
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   7 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 b 
                               
                               - 
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   15 
                                 
                               
                             
                             
                               2 
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     36 
                      
                     
                       - 
                     
                      
                     7 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       8 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   8 
                                 
                               
                               - 
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   4 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   b 
                                    
                                   
                                       
                                   
                                    
                                   16 
                                 
                               
                               - 
                               
                                 G 
                                 b 
                               
                             
                             
                               2 
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     38 
                      
                     
                       - 
                     
                      
                     8 
                   
                   ) 
                 
               
             
           
         
       
     
       The color-corrected plane of the green color elements can be obtained with Equation (37): 
       [0062]    
       
         
           
             
               
                 
                   G 
                   = 
                   
                     
                       
                         ( 
                         
                           
                             G 
                             r 
                           
                           + 
                           R 
                         
                         ) 
                       
                       + 
                       
                         ( 
                         
                           
                             G 
                             b 
                           
                           + 
                           B 
                         
                         ) 
                       
                     
                     2 
                   
                 
               
               
                 
                   ( 
                   37 
                   ) 
                 
               
             
           
         
       
     
         [0063]    In Step S 6 , the color-modified plane of the green color elements obtained in Step S 5  is used to correct the existing planes of the red color elements and the blue color elements respectively. The color-difference plane the between the planes of the red color elements and the green color elements and the color-difference plane between the planes of the blue color elements and the green color elements can be obtained with Equation (38): 
         [0000]        R   g   =R−G, B   g   =B−G    (38) 
         [0000]    wherein G is the color-corrected plane of the green color elements obtained in Step S 5 . Refer to  FIG. 13  a diagram schematically showing that a new color-difference plane R g  is worked out according to the method of the present invention. The new color-difference plane R g  can be obtained with Equation (39): 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       R 
                       g 
                     
                     = 
                     
                       
                         ∑ 
                         
                           j 
                           = 
                           1 
                         
                         8 
                       
                        
                       
                           
                       
                        
                       
                         
                           w 
                           j 
                         
                          
                         
                           R 
                           gj 
                         
                       
                     
                   
                   , 
                   
                     
 
                   
                    
                   
                     
                       w 
                       j 
                     
                     = 
                     
                       
                         e 
                         cj 
                       
                       
                         
                           ∑ 
                           
                             k 
                             = 
                             1 
                           
                           8 
                         
                          
                         
                             
                         
                          
                         
                           e 
                           ck 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   39 
                   ) 
                 
               
             
           
         
       
     
         [0000]    wherein e cj  j=1˜8 is the edge indicator of color difference. The edge indicators of color difference e cj  j=1˜8 can be obtained with Equations (40-1) to (40-8): 
         [0000]    
       
         
           
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       1 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   5 
                                 
                               
                               - 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 g 
                               
                               - 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   9 
                                 
                               
                             
                             2 
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     40 
                      
                     
                       - 
                     
                      
                     1 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       2 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                               - 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   6 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   10 
                                 
                               
                               - 
                               
                                 R 
                                 g 
                               
                             
                             
                               2 
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     40 
                      
                     
                       - 
                     
                      
                     2 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       3 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                               - 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   7 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   11 
                                 
                               
                               - 
                               
                                 R 
                                 g 
                               
                             
                             2 
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     40 
                      
                     
                       - 
                     
                      
                     3 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       4 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   8 
                                 
                               
                               - 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   4 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                 
                               
                               - 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   12 
                                 
                               
                             
                             
                               2 
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     40 
                      
                     
                       - 
                     
                      
                     4 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       5 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   5 
                                 
                               
                               - 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   13 
                                 
                               
                               - 
                               
                                 R 
                                 g 
                               
                             
                             2 
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     40 
                      
                     
                       - 
                     
                      
                     5 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       6 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                               - 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   6 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 g 
                               
                               - 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   14 
                                 
                               
                             
                             
                               2 
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     40 
                      
                     
                       - 
                     
                      
                     6 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       7 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                               - 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   7 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 g 
                               
                               - 
                               
                                 G 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   15 
                                 
                               
                             
                             
                               2 
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     40 
                      
                     
                       - 
                     
                      
                     7 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       8 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   8 
                                 
                               
                               - 
                               
                                 R 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   4 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 G 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   16 
                                 
                               
                               - 
                               
                                 G 
                                 g 
                               
                             
                             
                               2 
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     40 
                      
                     
                       - 
                     
                      
                     8 
                   
                   ) 
                 
               
             
           
         
       
     
         [0000]    Refer to  FIG. 14  a diagram schematically showing that a new color-difference plane B g  is worked out according to the method of the present invention. The new color-difference plane B g  can be obtained with Equation (41): 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       B 
                       g 
                     
                     = 
                     
                       
                         ∑ 
                         
                           j 
                           = 
                           1 
                         
                         8 
                       
                        
                       
                           
                       
                        
                       
                         
                           w 
                           j 
                         
                          
                         
                           B 
                           gj 
                         
                       
                     
                   
                   , 
                   
                     
 
                   
                    
                   
                     
                       w 
                       j 
                     
                     = 
                     
                       
                         e 
                         cj 
                       
                       
                         
                           ∑ 
                           
                             k 
                             = 
                             1 
                           
                           8 
                         
                          
                         
                             
                         
                          
                         
                           e 
                           ck 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   41 
                   ) 
                 
               
             
           
         
       
     
         [0000]    wherein e cj  j=1˜8 is the edge indicator of color difference. The edge indicators of color difference e cj  j=1˜8 can be obtained with Equations (42-1) to (42-8): 
         [0000]    
       
         
           
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       1 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   5 
                                 
                               
                               - 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 g 
                               
                               - 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   9 
                                 
                               
                             
                             2 
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     42 
                      
                     
                       - 
                     
                      
                     1 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       2 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                               - 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   6 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   10 
                                 
                               
                               - 
                               
                                 B 
                                 g 
                               
                             
                             
                               2 
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     42 
                      
                     
                       - 
                     
                      
                     2 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       3 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                               - 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   7 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   11 
                                 
                               
                               - 
                               
                                 B 
                                 g 
                               
                             
                             2 
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     42 
                      
                     
                       - 
                     
                      
                     3 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       4 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   8 
                                 
                               
                               - 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   4 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                 
                               
                               - 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   12 
                                 
                               
                             
                             
                               2 
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     42 
                      
                     
                       - 
                     
                      
                     4 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       5 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   5 
                                 
                               
                               - 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   13 
                                 
                               
                               - 
                               
                                 B 
                                 g 
                               
                             
                             2 
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     42 
                      
                     
                       - 
                     
                      
                     5 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       6 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                               - 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   6 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 g 
                               
                               - 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   14 
                                 
                               
                             
                             
                               2 
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     42 
                      
                     
                       - 
                     
                      
                     6 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       7 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                               - 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   7 
                                 
                               
                             
                             2 
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 g 
                               
                               - 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   15 
                                 
                               
                             
                             
                               2 
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     42 
                      
                     
                       - 
                     
                      
                     7 
                   
                   ) 
                 
               
             
             
               
                 
                   
                     e 
                     
                       c 
                        
                       
                           
                       
                        
                       8 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         1 
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   8 
                                 
                               
                               - 
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   4 
                                 
                               
                             
                             
                               2 
                                
                               
                                 2 
                               
                             
                           
                            
                         
                         + 
                         
                            
                           
                             
                               
                                 B 
                                 
                                   g 
                                    
                                   
                                       
                                   
                                    
                                   16 
                                 
                               
                               - 
                               
                                 B 
                                 g 
                               
                             
                             
                               2 
                             
                           
                            
                         
                       
                       ) 
                     
                     
                       - 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   
                     42 
                      
                     
                       - 
                     
                      
                     8 
                   
                   ) 
                 
               
             
           
         
       
     
       The color-corrected plane of the red color elements and the color-corrected plane of the blue color elements are obtained with Equation (43): 
       [0064]        R=G+R   g   , B=G+B   g    (43) 
         [0065]    In Step S 7 , Step S 5  and Step S 6  are repeated one to three times to obtain a color-corrected digital color image. 
         [0066]    Refer to  FIG. 15  for the process of from Step S 1  to Step S 4  and the results of the process. Refer to  FIG. 16  for the process of from Step S 5  to Step S 7  and the results of the process. The present invention utilizes the abovementioned heterogeneity-projection technology and the hard-decision rule to determine the optimal interpolation direction to reduce the color distortion and texture distortion of the digital image caused by the incorrect interpolation direction. Further, the present invention restores the color elements lost in the process that light passes through the CFA and makes the reproduced colors of the digital color image more close to the original colors. 
         [0067]    Refer to  FIGS. 17(A) to 17(D)  diagrams to compare the result of the method of the present invention with the results of other methods.  FIG. 17(A)  is the original image.  FIG. 17(B)  is the image demosaiced with the method proposed by Gunturk, and the PSNR (Peak Signal to Noise Ratio) thereof is 33.7776 dB.  FIG. 17(C)  is the image demosaiced with the method proposed by Lu, and the PSNR thereof is 32.2664 dB.  FIG. 17(D)  is the image demosaiced with the method proposed by the present invention, and the PSNR thereof is 34.9164 dB. Refer to  FIGS. 18(A) to 18(D)  further diagrams to compare the result of the method of the present invention with the results of other methods.  FIG. 18(A)  is the original image.  FIG. 18(B)  is the image demosaiced with the method proposed by Gunturk (2002), and the PSNR thereof is 31.9619 dB.  FIG. 18(C)  is the image demosaiced with the method proposed by Lu, and the PSNR thereof is 32.2929 dB.  FIG. 18(D)  is the image demosaiced with the method proposed by the present invention, and the PSNR thereof is 35.5103 dB. 
         [0068]      FIG. 19  shows the image demosaiced by Lu&#39;s method. In  FIG. 19 , one can see that several visible color artifacts are resided in the texture region of the demosaiced color image.  FIG. 20  shows the image demosaiced by Gunturk&#39;s method. In  FIG. 20 , one can see that several visible color artifacts are resided in the edge region of the demosaiced color image.  FIGS. 21 and 22  show the image demosaiced by the method of present invention. It is clear that the visible color artifacts resided in the demosaiced image are reduced efficiently. 
         [0069]    Those described above are the embodiments to exemplify the present invention to enable the persons skilled in the art to understand, make and use the present invention. However, it is not intended to limit the scope of the present invention. Therefore, any equivalent modification and variation according to the spirit of the present invention is to be also included within the scope of the claims of the present invention stated below.