Patent Publication Number: US-2009232395-A1

Title: Image processing device

Description:
RELATED APPLICATIONS 
     This application claims the benefit of Japanese Patent Application No. 2006-143668 filed on May 24, 2006 in Japan, the contents of which are incorporated herein by reference. 
     TECHNICAL FIELD 
     The present invention relates to image signal processing in a complementary-color single-chip camera, and particularly to an image processing device that improves horizontal resolution and reduces aliasing on a color border. 
     BACKGROUND ART 
       FIG. 7  is a configuration of a signal processing circuit of a complementary-color single-chip camera shown in Japanese Patent No. 3540758. Horizontal LPFs  103  and  104  having different frequency response characteristics comprise: a first HLPF  103  of a multitap configuration; and a second HLPF  104  of a few-tap configuration. The first HLPF  103  has the property of passing higher-frequency components than the second HLPF  104 . 
     An output of the first HLPF  103  is sent to a horizontal contour correction circuit  111 , which generates a horizontal contour signal (horizontal aperture signal) HAP. Likewise, an output of the second HLPF  104  is sent to a horizontal contour correction circuit  105 , which generates a horizontal contour signal (horizontal aperture signal) HAP. The first horizontal contour signal generated by the first horizontal contour correction circuit  111  and the second horizontal contour signal generated by the second horizontal contour correction circuit  105  are sent to a weighted addition circuit  112 . 
     Since the first HLPF  103  has the property of passing higher-frequency components than the second HLPF  104 , the first horizontal contour correction circuit  111  can generate a finer (higher-resolution) horizontal contour signal than the second horizontal contour correction circuit  105 . 
     In this regard, however, since the first HLPF  103  is formed of an LPF of a multitap configuration, the use of a horizontal contour signal generated by the first horizontal contour correction circuit  111  might cause ringing in a border area between colors in an image. The image processing device shown in  FIG. 7  is provided with: a chroma integrated value calculation circuit  301  for calculating a chroma integrated value based on a color difference signal outputted from a color signal processing circuit  202 ; and a weighting coefficient calculation circuit  302  for calculating a weighting coefficient for each pixel based on a chroma integrated value calculated for each chroma integration area by the chroma integrated value calculation circuit  301 . 
     Based on the weighting coefficient calculated for each pixel by the weighting coefficient calculation circuit  302 , the weighted addition circuit  112  calculates a weighted sum of the first and second horizontal contour signals. A high-resolution luminance signal can be obtained in both an achromatic color area and a chromatic color area by: using the first horizontal contour signal generated based on an output of the first HLPF  103 , which passes high-frequency components, for an achromatic color area that contains high-frequency components; and using the second horizontal contour signal, which does not cause ringing, for a chromatic color area that does not contain high-frequency components. 
     DISCLOSURE OF THE INVENTION 
     Problems to be Solved by the Invention 
     However, since the conventional image processing device requires a plurality of horizontal contour correction circuits, the scale of the circuit would expand. In addition, chroma adjusted with an edge value is used for the weighting coefficient control, and therefore an output signal of the LPF having a low frequency response characteristic is used for a chromatic color area, causing a problem that the resolution decreases in a chromatic color area as compared to an achromatic color area. 
     A purpose of the invention made in view of the above-mentioned background is to provide an image processing device capable of reducing the scale of the circuit and providing a high-resolution image even for a chromatic color area. 
     Means for Solving the Problems 
     An image processing device of the invention comprises: a plurality of horizontal low-pass filters for performing horizontal low-pass filtering on an inputted image signal, the plurality of horizontal low-pass filters having different frequency response characteristics; a weighted adder for calculating a weighted sum of output signals from the plurality of horizontal low-pass filters; and a horizontal contour correction unit for performing horizontal contour correction based on a weighted sum signal calculated by the weighted adder. 
     There are other aspects of the invention as described below. This disclosure of the invention therefore intends to provide part of the invention and does not intend to limit the scope of the invention described and claimed herein. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a block diagram of an image processing device of a first embodiment; 
         FIG. 2  is a graph of frequency response characteristics of LPFs of the first embodiment; 
         FIG. 3  is a conceptual drawing of pixels and image signals; 
         FIG. 4  is a block diagram of an image processing device of a second embodiment; 
         FIG. 5A  is a conceptual drawing of aliasing that occurs on a border between colors whose phases are opposite to each other for every line; 
         FIG. 5B  is a conceptual drawing of aliasing that occurs on a border between colors whose phases are the same phase for every line; 
         FIG. 6  is a block diagram of an image processing device of a third embodiment; and 
         FIG. 7  shows a configuration of a conventional image processing device. 
     
    
    
     BEST MODE OF EMBODYING THE INVENTION 
     The following is a detailed description of the invention. It will be understood that the embodiments described below are only examples of the invention, and the invention can be varied in various aspects. Therefore, the specific configurations and functions disclosed below do not limit the claims. 
     An image processing device of this embodiment comprises: a plurality of horizontal low-pass filters for performing horizontal low-pass filtering on an inputted image signal, the plurality of horizontal low-pass filters having different frequency response characteristics; a weighted adder for calculating a weighted sum of output signals from the plurality of horizontal low-pass filters; and a horizontal contour correction unit for performing horizontal contour correction based on a weighted sum signal calculated by the weighted adder. 
     In this configuration, since the horizontal contour correction process is performed after the weighted adder has calculated a weighted sum of output signals from the horizontal low-pass filters, it is not required to provide every one of the plurality of horizontal low-pass filters with a horizontal contour correction unit, and the scale of the circuit can be reduced. 
     The image processing device of the embodiment has: a feature value calculator for calculating hue from pixels surrounding a target pixel; and a weighting coefficient calculator for determining a weighting coefficient based on hue calculated by the feature value calculator, where the weighted adder calculates a weighted sum of the output signals using a weighting coefficient calculated by the weighting coefficient calculator. 
     In this configuration, by calculating a weighted sum using a weighting coefficient determined based on hue, whether priority is given to smooth image quality with little aliasing or to resolution can be chosen based on hue. For example, a process can be performed in which resolution is increased even for a chromatic area if the area has little change in hue. 
     The image processing device of the embodiment has: a feature value calculator for calculating saturation and hue from pixels surrounding a target pixel; and a weighting coefficient calculator for determining a weighting coefficient based on saturation, change in saturation, hue, and change in hue calculated by the feature value calculator, where the weighted adder calculates a weighted sum of the output signals using a weighting coefficient calculated by the weighting coefficient calculator. 
     In this configuration, by calculating a weighted sum using a weighting coefficient determined based on saturation and hue, whether priority is given to smooth image quality with little aliasing or to resolution can be chosen based on hue. For example, a process can be performed in which resolution is increased even for a chromatic area if the area has little change in hue. 
     The image processing device of the embodiment has: a plurality of horizontal low-pass filters to which an image signal and a delayed image signal are inputted, the plurality of horizontal low-pass filters having a same property; a vertical low-pass filter for performing vertical low-pass filtering on output signals from horizontal low-pass filters having a same frequency response characteristic; and a second weighted adder for calculating a weighted sum of an output signal of the vertical low-pass filter and an output signal of the horizontal low-pass filter, where the weighting coefficient calculator determines a second weighting coefficient to be used by the second weighted adder, based on hue calculated by the feature value calculator. 
     In this configuration, since a weighting coefficient for the second weighted adder is controlled according to hue, the degree of correction using the vertical low-pass filter can be controlled based on hue. On a color border, for example, aliasing that occurs on the color border can be reduced by using a greater amount of an output signal of the vertical LPF. 
     In the image processing device of the embodiment, the plurality of horizontal low-pass filters are separately placed on a plurality of lines; an image signal and a delayed image signal are inputted one by one to one of the plurality of lines according to a delay time thereof; and the second weighted adder calculates a weighted sum of an output signal of a central line of the plurality of lines and an output signal of the vertical low-pass filter. 
     In this configuration, when a weighted sum of output signals from low-pass filters having different frequency response characteristics is calculated, a time lag between the image signals whose weighted sum is to be calculated is eliminated, and an appropriate correction can be made. 
     The image processing device of the embodiment has: a border-area correction low-pass filter for correcting aliasing that occurs in a border area between colors, where when an image signal of a border area has been detected based on hue determined by the feature value calculator, the weighted adder calculates the weighted sum including an output signal from the border-area correction low-pass filter. 
     In this configuration, when an image signal of a color border area has been detected by a change in hue, aliasing that occurs on the border can be reduced by correcting using the border-area correction low-pass filter. The border-area correction low-pass filter is preferably a low-pass filter with a low frequency response characteristic so as to be able to correct aliasing on a color border. In other areas than a color border area, a high-resolution image can be obtained by not using the border-area correction low-pass filter. 
     Now, image processing devices of embodiments of the invention will be described in detail with reference to the drawings. 
     First Embodiment 
       FIG. 1  is a block diagram showing a configuration of an image processing device of a first embodiment of the invention. The image processing device comprises an image signal input unit  10 , 1HRAMs  12   a  and  12   b , LPFs  14  and  16 , an adder  18 , a feature value calculator  20 , a weighting coefficient calculator  22 , a weighted adder  24 , and a horizontal contour correction unit  26 . 
     In a field read signal that is inputted from the image signal input unit  10 , a mixed signal of Cy (cyan) and Mg (magenta) and a mixed signal of Ye (yellow) and G (green) are alternately aligned on (n) line, and a mixed signal of Cy (cyan) and G (green) and a mixed signal of Ye (yellow) and Mg (magenta) are alternately aligned on (n+1) line. As an image signal, an output signal of an imaging element to which CDS and an analog gain control are applied may be A/D converted and directly inputted; or image signals recorded in advance in a storage may be read one by one. 
     An image signal inputted from the image signal input unit  10  is delayed horizontal periods by the 1HRAMs  12   a  and  12   b . This allows image information of a plurality of lines to be vertically processed at the same time. While in the embodiment the two 1HRAMs  12   a  and  12   b  are used to process three lines at the same time, the configuration is not limited thereto, and a lot more 1HRAMs may be added to calculate a feature value with higher accuracy. 
     The LPFs  14  and  16  are filters for removing color components from an image signal. The LPF  14  is formed with a larger number of taps than the LPF  16 . 
       FIG. 2  shows frequency response characteristics of the LPFs  14  and  16 . As shown in  FIG. 2 , the LPF  14  has a frequency response characteristic of passing higher-frequency components as compared to the frequency response characteristic of the LPF  16 . 
     The adder  18  adds an input signal from the image signal input unit  10  and an output signal of the 1HRAM  12   b  together, calculates the average, and inputs the average to the feature value calculator  20  along with a signal of 1HRAM  12   a  (central line). 
     The feature value calculator  20  calculates an edge value, saturation, and hue from pixels surrounding a target pixel. The feature value calculator  20  calculates an edge value based on the amount of change in signals of the same color that are adjacent to each other with one pixel interval between them. The feature value calculator  20  determines saturation based on the absolute value of the difference between horizontally adjacent pixels. That is, saturation is calculated for each of a central, upper, and lower line signals, and the average is calculated. The averaging process is also performed horizontally, allowing the resulting value to be the saturation around a target pixel. The feature value calculator  20  determines hue based on a combination of change in horizontally adjacent signals of each of a central, upper, and lower line signals. 
       FIG. 3  shows an example in which the feature value calculator  20  determines hue. As shown in  FIG. 3 , if the signal levels of signals (Cy+Mg), (Ye+G), and (Cy+Mg) of a central line are “low,” “high,” and “low,” and if the signal levels of signals (Cy+G), (Ye+Mg), and (Cy+G) of the lines above and below the central line are “high,” “low,” and “high,” the feature value calculator  20  estimates that hue around the target pixel is close to green. 
     The hue calculation method is not limited to the adjacent difference scheme, but may be that hue is determined from a combination of pluses and minuses relative to an average value of surrounding signals for each of a central, upper, and lower line signals, or may be that hue is determined by synchronously detecting each of a central, upper, and lower line signals, assuming that signal variations by color components are 2-PSK. 
     The weighting coefficient calculator  22  determines a weighting coefficient k from an edge value, saturation, and hue calculated by the feature value calculator  20  and from the amounts of changes therein. The weighting coefficient calculator  22  first determines a coefficient c 1  that is proportional to saturation. That is, the weighting coefficient calculator  22  determines coefficient c 1  so that c 1  equals 0 for an achromatic color area, and that c 1  equals 1 for an area with high saturation. 
     The weighting coefficient calculator  22  then corrects coefficient c 1  with an edge value to determine a coefficient c 2 . This is for limiting the influence of a change in luminance that will be incorrectly calculated as a false color even in an achromatic area if the area is an edge area, since color information is determined from the amount of change in signals among a plurality of pixels by using the adjacent signal difference or other method. The weighting coefficient calculator  22  determines coefficient c 2  by subtracting an edge value multiplied by a prescribed value from coefficient c 1 . 
     The weighting coefficient calculator  22  then determines the amounts of changes in saturation and hue between pixels on the left and right of the target pixel. Since aliasing does not occur if there is no change in saturation or hue even in a chromatic color area, coefficient c 2  is decreased for an area with little change in saturation and hue and is set as weighting coefficient k which is to be determined Since aliasing is more obvious on a color border between hues, such as green and magenta, red and blue, and yellow and cyan, whose phases of signal variations by color components are opposite to each other, coefficient c 2  is increased and set as weighting coefficient k which is to be determined. 
     As above, weighting coefficient k is calculated in a range from 0 to 1 so that weighting coefficient k is set to 0 for an achromatic color area and, conversely, weighting coefficient k is set to 1 for an area whose saturation is high and whose saturation and hue are changing. 
     The weighted adder  24  determines a weighted sum signal by the following equation (1) in which weighting coefficient k calculated by the weighting coefficient calculator  22 , an output signal of the LPF  14 , and an output signal of the LPF  16  are used. 
       Weighted sum signal=(1 −k )*Output signal of  LPF  14 +k *Output signal of  LPF  16  (1) 
     In this way, by determining a weighted sum signal using weighting coefficient k that is determined according to a feature value, the usage ratio between the LPFs  14  and  16  can be changed according to a feature of an image. For example, a signal of the LPF  14 , which passes high-frequency components, is used for an achromatic color area or even for a chromatic color area if the area has little saturation and change in hue; and a signal of the LPF  16 , which causes little aliasing, is used for a chromatic color area with some saturation and change in hue. 
     The horizontal contour correction unit  26  receives an output signal of the weighted adder  24  and generates a horizontal contour correction signal. The process of the horizontal contour correction unit  26  is in principle a process of enhancing the amount of change in signal using an HPF, and the horizontal contour correction unit  26  performs a coring process for reducing the influence of noise, and the like. The above is a description of the image processing device of the embodiment. 
     In the image processing device of the embodiment, since horizontal contour correction is made after the weighted adder  24  has weighted output signals from the plurality of LPFs  14  and  16 , no more than one horizontal contour correction unit  26  is required. Consequently, the scale of the circuit of the image processing device can be reduced. 
     In the image processing device of the embodiment, by determining weighting coefficient k according to a value determined by the feature value calculator  20 , the weighting coefficient calculator  22  can change the usage ratio between the LPFs  14  and  16  according to a feature of an image. For example, a high-resolution image can be obtained for an achromatic color area and even for a chromatic color area if the area has little saturation and change in hue; and a high-quality image with little aliasing can be obtained for a chromatic color area with some saturation and change in hue. 
     Second Embodiment 
       FIG. 4  is a block diagram showing a configuration of an image processing device of a second embodiment. The configuration of the image processing device of the second embodiment is basically the same as that of the image processing device of the first embodiment, but is different in having a plurality of LPFs  16   a  to  16   c  having the same property. The image processing device of the second embodiment also has a vertical LPF (hereinafter referred to as the “VLPF”)  28  and a weighted adder  30 . 
     The plurality of LPFs  16   a  to  16   c  receives a signal from the image signal input unit  10  and output signals of the 1HRAMs  12   a  and  12   b , and performs low-pass filtering separately on the signals of the plurality of lines. The LPFs  16   a  to  16   c  have (1+Z −1 )/2 or other few-tap configurations to reduce the extent of aliasing on a color border at the expense of frequency response characteristics. In this regard, however, even an LPF with (1+Z −1 )/2 does not eliminate aliasing on a color border. 
     The VLPF  28  performs vertical low-pass filtering on output signals of the LPFs  16   a  to  16   c . In the second embodiment, the weighting coefficient calculator  22  calculates a weighting coefficient k 2  to be used by the weighted adder  30 , in addition to weighting coefficient k. 
     The weighted adder  30  calculates a weighted sum of a signal of the central line and an output signal of the VLPF  28  by the following equation (2) in which weighting coefficient k 2  is used. 
       Weighted sum signal=(1 −k 2)*Output signal of  VLPF+k 2*Signal of central line  (2) 
     In this regard, a relation between hue and weighting coefficient k 2  will be described. On a border between colors, such as green and magenta, and red and blue, whose phases of signal variations by color components are opposite to each other for every line, the VLPF  28  can reduce aliasing by performing vertical LPF though it decreases vertical resolution in the color border area. Therefore, in this case, weighting coefficient k 2  is set close to 0 so that a greater amount of an output signal of the VLPF  28  is used. On a border between colors, such as yellow and cyan, whose phases of signal variations by color components are the same phase for every line, the VLPF  28  cannot reduce aliasing, and therefore weighting coefficient k 2  is set close to 1 so that a greater amount of a signal of the center is used in order to avoid a decrease in vertical resolution. The above is a description of the image processing device of the second embodiment. 
       FIGS. 5A and 5B  show examples of aliasing that occurs on a color border. How aliasing occurs on a color border depends on hues on the left and right of the color border. On a border between colors, such as green and magenta, and red and blue, whose phases of signal variations by color components are opposite to each other for every line, patchy aliasing occurs on the color border as shown in  FIG. 5A . This is because phases of signal variations by color components differ in these hues between a signal of the central line and signals of the upper and lower lines, so that highs and lows of the signal level of aliasing occurring on the color border alternate line by line. 
     On a border between colors, such as yellow and cyan, whose phases of signal variations by color components are the same phase for every line, aliasing occurs in vertical stripes as shown in  FIG. 5B . Since phases of signal variations by color components are the same in these hues between a signal of the central line and signals of the upper and lower lines, highs and lows of the signal level of aliasing occurring on the color border is the same between a signal of the central line and signals of the upper and lower lines, so that aliasing occurs in vertical lines. Aliasing occurs more obviously on a color border between colors, such as green and magenta, red and blue, and yellow and cyan, whose phases of signal variations by color components are opposite to each other. 
     In the image processing device of the second embodiment, aliasing can be reduced by calculating a weighted sum of a signal of a central line and an output signal of the VLPF  28  with weighting coefficient k 2  depending on hue. That is, a high-resolution image can be obtained for an achromatic color area and even for a chromatic color area if the area has little saturation and change in hue; and a high-quality image with little aliasing can be obtained for a chromatic color area with some saturation and change in hue by using a signal on which vertical LPF has been performed. 
     Third Embodiment 
       FIG. 6  is a block diagram showing a configuration of an image processing device of a third embodiment of the invention. The configuration of the image processing device of the third embodiment is basically the same as that of the image processing device of the second embodiment, but the image processing device of the third embodiment further comprises an LPF  32  and a weighted adder  34 . 
     The LPF  32  has a tap configuration such as (1+3Z −1 +3Z −2 +Z −3 )/8. The LPF  32  has a further lower frequency response characteristic than the LPF  14  and the LPFs  16   a  to  16   c , and has a property of being able to limit aliasing on a color border better than the LPF  14  and the LPFs  16   a  to  16   c.    
     In the third embodiment, the weighting coefficient calculator  22  calculates a weighting coefficient k 3  to be used by the weighted adder  34 , in addition to weighting coefficients k and k 2 . 
     As described above, the VLPF  28  cannot reduce aliasing that occurs in vertical stripes on a border between hues such as yellow and cyan. In this case, weighting coefficient k 3  is set close to 0 so that a greater amount of an output signal of the LPF  32  is used. In cases other than this, weighting coefficient k 3  is set close to 1 so that a greater amount of an output signal of the weighted adder  30  is used in order to maintain horizontal resolution. That is, the LPF  32  is a filter for correcting aliasing in a border area and is used for a color border; while it is controlled not to substantially operate by setting weighting coefficient k 3  close to 1 in an area other than a color border area. 
     The weighted adder  34  calculates a weighted sum of an output signal of the LPF  32  and an output signal of the weighted adder  30  by the following equation (3) in which weighting coefficient k 3  is used. 
       Weighted sum signal=(1 −k 3)*Output signal of  LPF  32+ k 3*Output signal of weighted adder 30  (3) 
     As stated above, a high-resolution image can be obtained for an achromatic color area and even for a chromatic color area if the area has little saturation and change in hue; and a high-quality image with little aliasing can be obtained for a chromatic color area with some saturation and change in hue by using a signal on which vertical LPF has been performed depending on hue or by using a signal on which border-area correction LPF has been performed. 
     While there have been described what are at present considered to be preferred embodiments of the invention, it will be understood that various modifications and variations may be made thereto, and it is intended that appended claims cover all such modifications and variations as fall within the true spirit and scope of the invention. 
     INDUSTRIAL APPLICABILITY 
     The invention has a great advantage of being able to reduce the scale of the circuit and to provide a high-resolution image even for a chromatic color area, and is useful as a complementary-color single-chip camera or the like.