Abstract:
A technique for estimating an unsampled color data on the basis of plural color data sampled by a line image sensor. The sensor includes at least two types of color filters such as a red filter and a green filter. Those filters are alternately arranged pixel by pixel in a line of pixels so that A-type color data which corresponds to green data is sampled from a target pixel and plural B-type color data which correspond to red data are sampled from neighbor pixels neighboring the target pixel. An unsampled B-type color data of the target pixel is estimated by synthesizing the sampled A-type color data of the target pixel and the sampled B-type color data of the neighbor pixels.

Description:
This application is a continuation of application Ser. No. 07/764,425, filed Sep. 23, 1991, now abandoned. 
    
    
     BACKGROUND OF THE INVENTION 
     The present invention relates to an image reading apparatus. More particularly, it relates to an image reading apparatus comprising color line image sensors having a color filter for every pixel for use in color scanners and color copying machines. 
     Recent color scanners and digital color copying machines generally use line image sensors such as CCD&#39;s for reading images. 
     The line image sensors have a color filter arranged for every pixel to read color information. There have been many proposals to arrange the color filters. 
     FIG. 12 shows an example of a prior art line sensor arrangements. In FIG. 12, where R denotes a red pixel, G is a green pixel, and B is a blue pixel, a main scanning line has filters of the same color arranged therein, and two other scanning lines have different color filters arranged therein in a subscanning direction. 
     The arrangement described above has the advantage that the color filters can be fabricated and mounted rather easily and laterally long filters of the same colors may be easily prepared. 
     However, in the method shown in FIG. 12, as other color information with respect to a pixel exists in the subscanning direction, preceding and succeeding information has to be read as a scan is made. To do this, the number of the line memories has to be increased. 
     In order to solve such a problem, the inventors proposed a CCD line sensor in which, as shown in FIG. 11, red and green pixels were alternately arranged on a line, while blue pixels were continuously arranged on a different line. Note that a white filter instead of a blue filter may be used. In this case, the blue information can be obtained from the subtraction of the other two signals. 
     The arrangement of FIG. 11 is desirable because number of line memories can be reduced. However, the inventors found that it had the disadvantage that the resolution in the main scanning direction might be degraded as compared to that of the three-line method discussed above. As the red information, for example, is given for every other pixel, the resolution in the main scanning direction is halved. 
     The inventors estimated the lost information with use of a linear interpolation method illustrated in FIG. 3. In FIG. 3, where the order of a pixel of interest in the main scanning direction is denoted by i, green information G(i) on a green (G) channel can be obtained quickly. However, red information R(i) cannot be obtained. To overcome this, adjoining red informations R(i-1) and R(i+1) are brought in to make a mean signal R(i) of both signals. Of course, other informations R(i±2d+1, where d is an integer) can be used to increase the resolution. In any case, the other red informations are estimated from the known red information only. 
     After the inventors considered the linear interpolation method, they found that there is a limitation in the interpolation effect, particularly in reproducing a monochrome image in which false colors are mixed in the reproduced image, resulting in an increase of errors in a reproduced image compared with an original image. At a boundary where the region of a character changes to a non character region, for example, the original image changes from black to white as in a digital mode. However, the linear interpolation estimation does not change the information values of red and green to zero, but causes recognition of false colors. 
     SUMMARY OF THE INVENTION 
     In view of the foregoing, it is a general object of the present invention to increase image quality by reducing the reproduction error with a decreased number of necessary line memories. 
     In the present invention, the estimation is carried out in such a way that weighed coefficients are given to the color information obtained through the color filter for the pixel concerned and to the aiming color information obtained through color filters for at least two pixels adjoining the concerned pixel. In other words, in the estimation of R(i) mentioned above, the known green information G(i) in the same pixel is used. In this new method, the ordinary even interpolation technique is not used, but the interpolation considering the texture and the like of the image concerned and changing the interpolation characteristics as occasion may demand is carried out. 
     When the signal level changes in a digital mode in the information obtained through the color filter for the pixel concerned (and/or the information obtained from other pixels of the same color), and the pixel to be reproduced is judged to be monochrome, the estimation by interpolation is not made, but the actual information (intensity information) obtained through the pixel concerned is used for the reproduction. This can improve the reproducibility of images. 
     For estimating alternating red and green information as described above, a color synthesizing method is used. That is, an estimated value   (x) can be given by the following equation. 
     
          (x)=Σ.sub.j [f.sub.j (x)*a.sub.ij (x)] 
    
     with j being an unsampled color component data of the target pixel to be estimated, where i and j are any one of R, G and B, and where two or more colors are used for estimation. A coefficient a ij  can be calculated with use of a determinant that minimizes the mean square error of the actual image value and estimated value. 
     The ordinary linear interpolation method uses only one color information obtained through the red pixel (red filter) to estimate the red information. Instead, the color synthesizing method according to the present invention does not only use the red information, but also uses additional information obtained through the green pixel at a position at which interpolation is made (plus other green pixels as needed), and the coefficient a ij  is determined to minimize the mean square error on the basis of the information obtained through at least two color filters. 
     For example, when the information obtained through the green pixel to be interpolated can be determined to be monochrome as the intensity changes in a digital mode, the coefficient for the red information is made to be zero to prevent a false red color to be caused and to reproduce the image according to the intensity with the only use of the green information. 
     As described above, the image reading apparatus according to the present invention can improve the image quality in the way that considering the characteristics of an image to be reproduced, a better approximate solution to the original image is made by changing the coefficients so that the characteristics can be reproduced as close as possible with use of at least two items of color information. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     FIG. 1 is a first embodiment illustrating the major sections of the image reading apparatus according to the present invention. 
     FIG. 2 is a timing chart illustrating the operation in the first embodiment given in FIG. 1. 
     FIGS. 3 and 4 illustrate principles of the present invention. 
     FIG. 5 illustrates principles of the color synthesizing method of the present invention. 
     FIG. 6 illustrates the definitions of the functions used in FIG. 5. 
     FIG. 7 illustrates contents of the mean square error. 
     FIG. 8 illustrates contents of a determinant for obtaining a value for minimizing the error e given in FIG. 7. 
     FIG. 9 illustrates contents of the determinant used in a computer simulation. 
     FIG. 10 illustrates effects of the embodiments of the present inventions. 
     FIG. 11 illustrates a pixel arrangement of a CCD line image sensor used in the present invention. 
     FIG. 12 illustrates a pixel arrangement of an ordinary three-line CCD line sensor. 
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
     FIG. 4 illustrates principles of the present invention where d indicates a pitch between pixels, and f r  (X) is the density of red R at the pixel X. Let a channel at position 0 be green. f g  (0) is known. On the other hand, f r  (0) itself cannot be obtained in terms of data, and has to be estimated by analogy. Red information f r  (-d) and f r  (d) at both adjoining pixels are already known. Assuming f r  (0) can be obtained on the analogy of the three data, the red information can be obtained by following equation. (See FIG. 4). ##EQU1## 
     where a r  (-d) is a coefficient for the red information at a position -d, and a g  (0) is a coefficient for the green information at a position 0. 
     To obtain these three coefficients a r  (+d), a r  (-d), and a g  (0) is the objective in the invention. 
     In a simple example where a r  (-d)=a r  (d)=1/2 and a g  (0)=0, mere linear interpolation is used, but the green information is not used. 
     On the other hand, for a g  (0)=1, only the green information for the pixel concerned is used. This occurs, for example, in monochrome image data and the like. 
     As the coefficients are changed by the image information in that way, better approximate solutions can be obtained. 
     The following describes the method for determining the coefficient with use of the image information. Assuming the coefficients can be obtained in Eq. (1), the approximation can be made when the difference between the estimated coefficient value f r  and an unknown actual value f r  is small. In other words, the coefficients should be determined so that the mean square error of the estimated value and the actual value will be minimum. This means that the coefficients should be derived under a restricted condition that the mean square error partially differentiated by the respective coefficients should be made zero. 
     
         e.sub.r.sup.2 =E[ -f.sup.r).sup.2 ] 
    
     where E is a set mean. The coefficients should be determined so that the result can be made minimum. 
     Therefore, the following equation should be solved. ##EQU2## 
     Then, the following determinant can be obtained. ##EQU3## 
     wherein R is a correlation set mean, which is 
     
         Rf.sup.r f.sup.r (t)=E[f.sup.r (X)f.sup.r (x-t)] 
    
     This is a set mean of autocorrelation coefficients for a distance t of a certain section X. 
     Similarly, we can obtain 
     
         Rf.sup.r f.sup.g (t)=E[f.sup.g (X)f.sup.r (x-t)] 
    
     
         Rf.sup.g f.sup.g (t)=E[f.sup.g (X)f.sup.g (x-t)] 
    
     By calculating the determinant mentioned above, we can obtain the concerned coefficients a r  (+d), a r  (-d), and a g  (0). 
     The correlation coefficient on the left side of the determinant can be obtained by calculation. As an example, Rf r  f r  (2d) at the first row and second line is an autocorrelation coefficient for a distance 2d. As red appears every two pixels, the product of multiplication of the values of adjacent red pixels should be averaged for a certain section, say, for one main scanning line. 
     However, the right side terms cannot be obtained as it is. For example, Rf r  f r  (d) is the autocorrelation coefficient for the distance d, but cannot be directly obtained because of the fact that the red pixel exists every two (2d)s. 
     As Rf r  f r  (0) and Rf r  f r  (2d) are known, the autocorrelation coefficient is obtained by estimation. It is found that a better approximation can be made by making an analogy between the correlation coefficient and specified function. 
     Similarly, as Rf r  f g  (0) cannot be obtained, this is made by analogy from the known Rf g  f r  (d) or the like. The other coefficients can be obtained in the way described above. 
     It will be understood that the obtaining of the coefficients is not limited to the red channel discussed above as an example, but extends to the green channel. 
     As described above, the coefficients can be calculated, and the estimated values are fed out to a write system. 
     FIG. 1 shows an example of major sections of the image reading apparatus executing the calculations described above. The red and green data accumulated in a line sensor 1 is fed out through a timing adjusting buffer memory 4 and a shift register 6. The output signal is amplified through an analog amplifier 11, and is converted to digital data by an A/D converter 13. On the other hand, the blue data accumulated in a line sensor 2 is fed out through a shift register 8. The output signal is amplified through an analog amplifier 12, and is converted to digital data by an A/D converter 14. FIG. 1 also shows transfer control gates 3, 5, 7, 9, and 10. 
     Red, green, and blue latches RL1, GL1, BL1, and BL2 are provided to combine the three color data. If clocks CLK1 through CLK5, as shown in FIG. 2, are input, combinations of R, G, and B are output at times t1, t2, t3, and t4, as shown. The respective combinations are temporarily stored in color line memories 15, 16, and 17. 
     The process is made to proceed for every line. As the data are stored in the line memories 15, 16, and 17, product calculation of adjacent terms are sequentially made for correlation coefficient calculation by a coefficient circuit 18. When the product calculation is completed for one line, set means of the correlation coefficients are calculated. Data estimation is executed to derive necessary coefficients a, b, and c corresponding, respectively, to a r  (-d), a r  (d), and a g  (0). 
     In turn, data before one line is read from the line memories 15 and 16 at the next line reading instant. Calculation is made in terms of the obtained coefficients a, b, and c by an integrator 19. The final result is fed out to a write system 20. At the same time, the blue data also are input to the write system 20. Such a calculation process is made for every line to obtain desired results. Note that FIG. 1 illustrates the analogy of the red data at the position of the green pixel G1, and indicates the signal process flows by dotted arrows. 
     In the above-described embodiment, the two adjacent pixels are taken as an example. As should be apparent, a larger number of pixels can be treated using the same theory. 
     In the above-described embodiment, the set mean is taken on the whole line. Alternatively, calculation can be made on sectioned areas. In this way, the coefficients can be changed for better approximation even if the texture of the image changes on a line, for example, in the case that the right half of the image has a photograph and the left half of the image has characters. This provides far better approximation. In this case, the set mean has to be taken only on necessary areas while a product of adjacent terms is always being calculated. 
     As described so far, the image reading apparatus of the present invention has the advantage of obtaining information closer to the original image than the usual simple linear interpolation method. This is due to interpolation with use of other color information as occasion may demand, without fixing the interpolation coefficients. 
     As a result of a simple experiment made by the inventors, it was confirmed that the mean square error of the output image with the original character image could be decreased to a great extent as compared with the simple linear interpolation. 
     In turn, a method of determining coefficients as abovementioned is further explained with reference to the accompanying drawings. 
     FIG. 5 illustrates the principles of the embodiment. FIG. 6 illustrates the definitions of functions used in FIG. 5. 
     Let the width of red (R) and green (G) pixels be denoted by d as shown in FIG. 11. The red and green information, as shown on the upper row in FIG. 6, are obtained every 2d, respectively. 
     In FIG. 5, f s   r  (X) and f s   g  (X) are defined as shown in the middle of the figure with use of original values f r  and f g , noise terms n r  and n g , and a weight coefficient δ, where s means a sample value. 
     In the embodiment, as shown in the upper curves in FIG. 5, interpolation for red, for example, for X=0, should be made by use of not only the red information on both sides separated d (plus other red pixel information as needed), but also at least the green information at the position concerned (plus other red pixel information as needed). 
     Estimated values by interpolation, as shown in FIG. 5, are shown in the summation of the multiplications of the interpolation coefficients (a r  and a g  or b r  and b g ) and the functions concerning the information of R and G defined in FIG. 6. The interpolation coefficients a r  and a g  or b g  and should be determined so that the mean square errors e r   2  and e g   2  of the estimated values   and   and the actual values can be minimized (FIG. 7). 
     Solutions that minimize the mean square errors can be obtained by solving the determinant given in FIG. 8, or by multiplying the inverse matrix from the left side. 
     In order to compare effects of the color synthesization in the embodiment with those of the ordinary linear interpolation, the inventors made a computer simulation under the following conditions. 
     Data: Color image of 256×256 pixels. 
     Sampling rate: 2 pixels (Size of the R and G sensors is 1 pixel each.) 
     Sampled data: R and G. 
     Estimation was made both for the color synthesization in the embodiment and the ordinary linear interpolation. The correlation coefficients R f   r   f   r  (S) and R f   r   f   g  (S) were obtained by ##EQU4## 
     The correlation coefficients were estimated with exponential functions. 
     FIG. 9 shows a determinant used in the simulation. 
     FIG. 10 shows the results of the simulation. 
     It was proved that the mean square error e for one pixel in the color synthesization was less than that of the linear interpolation even when the kind of the image concerned was changed. This means that the reproducibility in the embodiment is higher that of the linear interpolation. 
     The image reading apparatus according to the present invention is effective, particularly in the color pixels having high correlation among them. It is not only actually effective for reducing the false colors in reproducing the monochrome image, but it is also effective for reducing error in reproducing the color image. 
     As described above in detail, the image information estimation with use of the color synthesizing method of the present invention has the advantage of improving the image quality with the false colors reduced in the reproduced image even if the simplified two-line linear image sensor is used. The advantage is accomplished in the way that interpolation of one color has other color information added, and the interpolation coefficients are changed to synthesize information according to the information so that the error can be minimized.