Patent Publication Number: US-9426437-B2

Title: Image processor performing noise reduction processing, imaging apparatus equipped with the same, and image processing method for performing noise reduction processing

Description:
TECHNICAL FIELD 
     The present invention relates to an image processor configured to perform noise reduction on image data acquired by an imaging device during shooting, an imaging apparatus equipped with the same, and an image processing method. 
     BACKGROUND ART 
     In an imaging device (single-plate solid-state imaging device) of a digital camera or the like, one color filter is generally provided for each photodiode to form a color filter array called a Bayer array, for example. Image data acquired by the imaging device is data (RAW data) having only color information on one color in each pixel, and can be seen as an image by interpolating color information lacking in each pixel from peripheral pixels. 
     For a subject having a frequency higher than a pixel spacing (pixel pitch) of the imaging device, failure to perform correct sampling causes moire and false colors. Particularly, in the imaging device having the Bayer array, a ratio of pixels of each of R and B to pixels of G is 1/2, leading to more likelihood of false colors. 
     As a general countermeasure to prevent this, an optical low-pass filter is inserted in front of the imaging device to lower the frequency of the subject. However, in this case, there are negative effects such as degraded resolution of the subject and cost increase caused by the optical low-pass filter. 
     Therefore, there has been proposed a method of reducing false colors by performing edge detection through edge detection filter processing on luminance components, detecting a false color by determining whether or not color-difference components are within a predetermined range, and then smoothing or taking an arithmetic average between false-color pixels and peripheral pixels in a region where the false color is detected (see, e.g., Patent Literature 1: Japanese Patent No. 4035688). 
     There has also been proposed a method of reducing false colors by taking two images at a focal position and a non-focal position of a lens, detecting false colors based on a difference between the two images, and then changing a mixture ratio of the two images (see, e.g., Patent Literature 2: Japanese Patent Application Publication No. 2011-109496). 
     However, among the above conventional technologies, when the processing is performed in a limited area for the false-color detected pixels and the peripheral pixels as in the case of Patent Literature 1, a special circuit configuration is required. This makes it difficult for a general-purpose image processing IC to perform the processing. 
     Also, when two images are taken as in the case of Patent Literature 2, a time lag occurs in shooting between the two images. For this reason, mixing of the two images also requires special processing such as extraction of feature points and alignment therebetween. 
     SUMMARY OF INVENTION 
     It is an object of the present invention to provide an image processor capable of reducing false colors generated in an image with a simple process configuration, and also to provide an imaging apparatus equipped with the same and an image processing method. 
     In order to achieve the above object, an aspect of the invention includes: a first image creator which creates first noise-processed image data by performing noise reduction processing for data on one image acquired by an imaging device; a second image creator which creates second noise-processed image data by performing noise reduction processing corresponding to a false color generated in the first noise-processed image data; and an image synthesizer which reduces false colors by synthesizing the first noise-processed image data and the second noise-processed image data. 
    
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         FIGS. 1A to 1C  show a digital camera, and  FIG. 1A  is a top view of the digital camera,  FIG. 1B  is a front view of the digital camera, and  FIG. 1C  is a back view of the digital camera. 
         FIG. 2  is a block diagram of a control system of the digital camera. 
         FIG. 3  is a diagram showing a general process flow of image processing. 
         FIG. 4A  is a diagram showing an example of a γ correction curve, and  FIG. 4B  is a diagram showing characteristics of a general output device such as an LCD and a CRT. 
         FIG. 5  is a diagram showing characteristics of color correction in a CbCr color space. 
         FIG. 6  is a diagram showing an internal configuration of a device to perform edge enhancement processing. 
         FIG. 7  is a diagram showing a process flow in Embodiment 1 to process RAW data. 
         FIGS. 8A to 8C  are diagrams showing positions to perform noise reduction,  FIG. 8A  showing a state before interpolation,  FIG. 8B  showing a state after interpolation of R information, and  FIG. 8C  showing a state after YCbCr conversion of Cr information. 
         FIGS. 9A to 9C  are diagrams showing a low-pass filter used for noise reduction,  FIG. 9A  showing the case where the low-pass filter is at a medium level,  FIG. 9B  showing the case where the low-pass filter is at a strong level, and  FIG. 9C  showing the case where the low-pass filter is at a weak level. 
         FIG. 10A  is a diagram showing an edge extraction filter,  FIG. 10B  is a diagram showing an edge extraction filter ( 1 ) before the interpolation, and  FIG. 10C  is a diagram showing an edge extraction filter ( 2 ) before interpolation. 
         FIG. 11A  is a diagram showing brightness data,  FIG. 11B  is a diagram showing G data, and  FIG. 11C  is a diagram showing data before interpolation. 
         FIGS. 12A and 12B  are diagrams showing a color detection method,  FIG. 12A  showing detection of hue and chroma and  FIG. 12B  showing detection of brightness. 
         FIG. 13  is a diagram showing an example of NR level setting for a false color generation level (FCL) range. 
         FIG. 14A  is a diagram showing an image synthesis ratio table,  FIG. 14B  is a diagram showing a false color generation level, and  FIG. 14C  is a diagram showing an image synthesis table. 
         FIG. 15  is a diagram showing a process flow in Embodiment 2 to process JPEG data. 
     
    
    
     DESCRIPTION OF EMBODIMENTS 
     Embodiments of the present invention will be described below with reference to the drawings. 
     Embodiments 
       FIGS. 1A to 1C  show a digital camera (imaging apparatus), and  FIG. 1A  is a top view of the digital camera,  FIG. 1B  is a front view of the digital camera, and  FIG. 1C  is a back view of the digital camera. 
     As shown in  FIG. 1 , the digital camera has a sub-LCD (liquid crystal display)  1 , a release button  2  and a shooting/play switching dial  4  on its top. The sub-LCD  1  is a display unit to display the maximum number of images that can be shot, and the like, for example. 
     On the front of the digital camera, a strobe light emitting part  3 , a ranging unit  5 , a remote-control light receiving part  6 , a barrel unit  7  and an optical finder (front side)  11   a  are provided. Moreover, on the back of the digital camera, an AF LED (autofocus light emitting diode)  8 , a strobe LED  9 , an LCD monitor  10 , an optical finder (back side)  11   b , a zoom button  12 , a power switch  13 , and an operation unit  14  are provided. 
     Also, on the side of the digital camera, a memory card throttle  121  is provided, into which a memory card  130  (see  FIG. 2 ) is fitted. 
       FIG. 2  is a block diagram of a control system of the digital camera. In  FIG. 2 , reference numeral  104  denotes a camera processor. The camera processor  104  is a controller including a CCD 1  signal processing block  1041 , a CCD 2  signal processing block  1042 , a CPU block  1043 , a Local SRAM  1044 , a USB block  1045 , a serial block  1046 , a JPEG/CODEC block  1047 , a RESIZE block  1048 , a TV signal display block  1049 , and a memory card controller block  10410 . These blocks and the like are connected to each other through a bus line. 
     Moreover, outside the camera processor  104 , there is provided an SDRAM  103  to store RAW-RGB image data, YUV image data and JPEG image data. The SDRAM  103  is connected to the camera processor  104  through a memory controller (not shown) and the bus line. 
     Outside the camera processor  104 , there are further provided a RAM  107 , an embedded memory  120  and a ROM  108  storing control programs, parameters and the like, all of which are also connected to the camera processor  104  through the bus line. 
     A strobe circuit  114  having the strobe light emitting part  3 , the ranging unit  5  and a temperature sensor  124  are connected to the CPU block  1043  in the camera processor  104 . 
     The strobe light emitting part  3  and the strobe circuit  114  are devices to supplement for the amount of light when the amount of light such as natural light is not enough. In shooting in a dark place or shooting a dark subject, the CPU block  1043  in the camera processor  104  transmits a strobe light emitting signal to the strobe circuit  114 , with which the strobe circuit  114  causes the strobe light emitting part  3  to emit light to illuminate the subject. 
     The ranging unit  5  is a device to measure a distance between the camera and the subject. The current digital camera uses a CCD-AF system for focusing by detecting a contrast of an image formed in a CCD  101  that is an imaging device and then moving a lens to a position with the highest contrast. However, the CCD-AF system has a problem that a focusing operation is slow because the contrast is detected by moving the lens little by little. 
     To solve this problem, there has been an attempt to speed up the focusing operation by constantly acquiring information on the distance from the subject with the ranging unit  5  and instantly moving the lens based on the distance information. 
     The temperature sensor  124  is a device to measure the ambient temperature. The temperature sensor  124  measures the temperatures inside and outside the camera and turns off the power of the camera in case of an abnormal temperature rise, or changes the control of the camera by referring to data acquired by the temperature sensor  124 . 
     The barrel unit  7  includes: a zoom optical system  71  having a zoom (ZOOM) lens  71   a  to capture an optical image of the subject, and a zoom motor  71   b ; a focus optical system  72  having a focus (FOCUS) lens  72   a  and a focus motor  72   b ; an aperture stop unit  73  having an aperture stop  73   a  and an aperture stop motor  73   b ; a mechanical shutter unit  74  having a mechanical shutter  74   a  and a mechanical shutter motor  74   b ; and a motor driver  75  to drive the motors. 
     The motor driver  75  is controlled to be driven according to a drive command from the CPU block  1043  in the camera processor  104  based on an input signal to the remote-control light receiving part  6  or an operation input to an operation Key unit SW 1  to SW 13  (the operation unit  14  in  FIG. 1 ). 
     The ROM  108  stores a control program and parameters for control, which are described in a code readable by the CPU block  1043 . When the power switch  13  (see  FIG. 1 ) is operated to turn on the power of the digital camera, the program is loaded into an unillustrated main memory. Accordingly, the CPU block  1043  controls operations of the respective units according to the program, and temporarily stores data required for control and the like in the RAM  107  and the Local SRAM  1044  in the camera processor  104 . Using a rewritable flash ROM as the ROM  108  enables the control program and the parameters for control to be changed, thereby facilitating function version upgrade. 
     The CCD  101  is a solid-state imaging device to perform photoelectric conversion on optical images. Also, an F/E (front end)-IC  102  includes a CDS  1021  to perform correlation double sampling for image noise removal, an AGC  1022  to perform gain adjustment, an A/D  1023  to perform digital signal conversion, and a TG  1024 . A vertical synchronizing signal (VD signal) and a horizontal synchronizing signal (HD signal) are supplied to the TG  1024  by the CCD 1  signal processing block  1041 . Moreover, the TG  1024  generates a drive timing signal for the CCD  101  and the F/E-IC  102 , which are controlled by the CPU block  1043 . 
     The CCD 1  signal processing block  1041  in the camera processor  104  performs white balance setting and gamma setting for output data from the F/E-IC  102 , which is imaging data acquired by the CCD  101 , and supplies the VD signal and the HD signal to the TG  1024  as described above. The CCD 2  signal processing block  1042  performs filtering processing to convert data into brightness data or color-difference data. 
     The CPU block  1043  controls the operations of the respective units described above. The Local SRAM  1044  temporarily stores the data required for control and the like described above. The USB block  1045  performs USB communication with an external device such as a personal computer. The serial block  1046  performs serial communication with the external device such as the personal computer. 
     The JPEG/CODEC block  1047  performs JPEG compression and extension. The RESIZE block  1048  increases and reduces the size of the image data by interpolation. The TV signal display block  1049  converts the image data into video signals for display on an external display device such as a liquid crystal monitor and a TV. The memory card controller block  10410  controls the memory card  130  to record the shot image data. 
     The SDRAM  103  temporarily stores the image data when the image data is subjected to various processing by the camera processor  104 . The image data to be stored includes “RAW-RGB image data” acquired by the CCD  101  through the F/E-IC  102  and subjected to the white balance setting and gamma setting by the CCD 1  signal processing block  1041 , “YCbCr image data” subjected to brightness data/color-difference data conversion by the CCD 2  signal processing block  1042 , “JPEG image data” subjected to JPEG compression by the JPEG/CODEC block  1047 , and the like, for example. 
     A voice recording unit  115  is connected to the CPU block  1043  in the camera processor  104 . The voice recording unit  115  includes a microphone  1153  to receive voice signals from a user, a microphone AMP  1152  to amplify the received voice signals, and a voice recording circuit  1151  to record the amplified voice signals. 
     Also, a voice reproduction unit  116  is connected to the CPU block  1043 . The voice reproduction unit  116  includes a voice reproduction circuit  1161  to convert the recorded voice signals into signals that can be outputted from a speaker  1163 , an audio AMP  1162  to amplify the converted voice signals and to drive the speaker  1163 , and the speaker  1163  to output the voice signals. 
     A SUB-CPU  109  is connected to the CPU block  1043 . The SUB-CPU  109  is a CPU having a ROM and a RAM embedded in one chip. The SUB-CPU  109  outputs output signals from the operation Key unit SW 1  to SW 13  or the remote-control light receiving part  6  to the CPU block  1043 , as user operation information. Also, the SUB-CPU  109  converts the state of the camera outputted by the CPU block  1043  into a control signal for the sub-LCD  1 , the AF LED  8 , the strobe LED  9  and a buzzer  113 , and outputs the control signal. 
     The sub-LCD  1  is a display unit to display the maximum number of images that can be shot, and the like, for example. The LCD driver  111  is a drive circuit to drive the sub-LCD  1  according to the output signal from the SUB-CPU  109 . 
     The AF LED  8  is an LED to display a focusing status in shooting. The strobe LED  9  is an LED to display a strobe charging status. Note that the AF LED  8  and the strobe LED  9  may be used for another display purpose such as displaying a state where the memory card is being accessed. The operation Key unit SW 1  to SW 13  is a Key circuit to be operated by the user. The remote-control light receiving part  6  is a receiver to receive a signal from a remote-control transmitter operated by the user. 
     A USB connector  122  is connected to the USB block  1045 . The USB connector  122  is a connector for USB connection with the external device such as the personal computer. Also, a serial driver circuit  1231  is connected to the serial block  1046 . The serial driver circuit  1231  performs voltage conversion on an output signal from the serial block  1046  to perform serial communication with the external device such as the personal computer. An RS-232C connector  1232  for serial connection with the external device such as the personal computer is connected to the serial driver circuit  1231 . 
     An LCD driver  117  and a video AMP  118  are connected to the TV signal display block  1049 . Among them, the LCD driver  117  is a drive circuit to drive the LCD monitor  10  (see  FIG. 1 ) and has a function to convert a video signal outputted from the TV signal display block  1049  into a signal to be displayed on the LCD monitor  10 . The user can monitor the state of the subject before shooting, check the shot images or check the image data recorded in the memory card  130  and the embedded memory  120  by viewing the contents displayed on the LCD monitor  10 . 
     Meanwhile, the video AMP  118  is an amplifier to convert the video signal outputted from the TV signal display block  1049  to 75Ω impedance. A video jack  119  is a jack for connection with the external display device such as the TV. 
     The memory card throttle  121  is connected to the memory card controller block  10410 . The memory card throttle  121  is a throttle for mounting the detachable memory card  130 . The embedded memory  120  is a memory to enable the shot image data to be stored even when the memory card  130  is not fitted into the memory card throttle  121  described above. 
     Next, a general flow of image processing will be described.  FIG. 3  shows a general process flow of image processing. This process flow is included in the CCD 1  signal processing block  1041  and the CCD 2  signal processing block  1042  shown in  FIG. 2 . Data obtained by subjecting a signal outputted from the CCD  101  to sampling for each pixel and then to A/D conversion is generally called RAW data because the data is yet to be subjected to image processing. Data to be inputted to an image processing unit is the RAW data. 
     The general flow of image processing is executed in the order of WB (white balance) processing, γ (gamma) correction processing, interpolation processing, YCbCr conversion processing, color correction processing and edge enhancement processing. These processings will be described in detail below. 
     [WB (White Balance) Processing] 
     A color filter of any one of colors, RED, GREEN and BLUE, is attached to each pixel of a CCD photodiode which accumulates light from the subject. However, since the amount of transmitted light varies depending on the color of the filter, the amount of charges to be accumulated in the photodiode also varies. GREEN is most sensitive, and RED and BLUE are less sensitive than GREEN, i.e., about half as sensitive as GREEN. In the WB processing, RED and BLUE are multiplied by a gain to cover such a difference in sensitivity and to make white in a shot image look whiter. Also, the color of an object varies depending on a light-source color (e.g., sunlight, fluorescent light or the like). Thus, even when a light source is changed, the gain of RED and BLUE is changed and controlled so as to make white look whiter. 
     [γ (Gamma) Correction Processing] 
       FIG. 4A  shows an example of a γ correction curve. The horizontal axis indicates an input signal and the vertical axis indicates an output signal. In this processing, non-linear input-output conversion is performed. Generally, in an output device such as an LCD and a CRT, output is given in a non-linear pattern with respect to input as shown in  FIG. 4B . In the case of such a non-linear output, the brightness has no gradation and the image turns out to be dark. For this reason, a person cannot correctly see the image. Thus, the γ correction processing is performed considering the characteristics of the output device to preprocess the input signal so that the output maintains its linearity with respect to the input. 
     [Interpolation Processing] 
     The CCD has an array called a Bayer array, in which a color filter of any one of the colors, RED, GREEN and BLUE, is attached to each pixel. The RAW data only has information on one color in one pixel. However, in order to view the RAW data as an image, the data needs to have information on the three colors, RED, GREEN and BLUE, in one pixel. Therefore, interpolation processing is performed to interpolate information on lacking two colors from peripheral pixels. 
     [YCbCr Conversion Processing] 
     While an RGB data format of the three colors, RED, GREEN and BLUE, is used at the RAW data stage, conversion to a YCbCr data format of a luminance signal Y and a color-difference signal CbCr is performed in YCbCr conversion processing. As to a JPEG image having a file format generally used for a digital camera or the like, an image is created from YCbCr data. Thus, the RGB data is converted into YCbCr data. The conversion equations are as follows.
 
 Y= 0.299× R+ 0.587× G+ 0.114× B  
 
 Cb=− 0.299× R− 0.587× G+ 0.886× B  
 
 Cr= 0.701× R− 0.587× G− 0.114× B  
 
     Here, R denotes RED, G denotes GREEN and B denotes BLUE. 
     [Color Correction Processing] 
     Color correction includes chroma setting, hue setting, partial hue change setting, color suppression setting, and the like. The chroma setting is parameter setting to determine the color density.  FIG. 5  shows a CbCr color space. For example, for the color of RED, the longer the vector length from the origin to the RED dot in the second quadrant, the higher the color density. 
     Next, the hue setting is parameter setting to determine the shades of color. For example, even if the vector length is the same for the color of GREEN in the third quadrant in  FIG. 5 , the shades of color vary when the direction of the vector is changed. 
     The partial hue change setting is a setting to rotate a partial color region in the direction of arrow A as shown in the fourth quadrant in  FIG. 5 . When the chroma is strong, the color density is increased and, at the same time, color noise tends to be increased. 
     Thus, in the color suppression setting, a threshold is set for the luminance signal, for example, and color noise suppression control is performed by suppressing the chroma in a region lower or higher than the threshold. 
     [Edge Enhancement Processing] 
     As shown in  FIG. 6 , the edge enhancement processing includes: an edge extraction filter to extract an edge portion from the luminance signal Y of the image; a gain multiplier to multiply the edge extracted by the edge extraction filter by a gain; a low-pass filter (LPF) to remove image noise in parallel with the edge extraction; and an adder to add the edge extraction data after the gain multiplication and the image data after the LPF processing. The edge strength is determined by the gain used by the gain multiplier. The edge is strong when the gain is large, while the edge is weak when the gain is small. 
     The edge enhancement processing is an important parameter because the edge detection direction or edge extraction amount also varies depending on a filter coefficient of the edge extraction filter. With the filter coefficient of the LPF, the image noise is reduced by smoothing the image. While the stronger the LPF, the less the noise, fine portions are crushed by the smoothing, resulting in a likelihood of losing the resolution. 
     [Other Processing] 
     The image processing also includes resize processing of changing to the image size to be stored, JPEG compression processing of compressing the information amount, and the like. 
     Embodiment 1 
     Next, description will be given of Embodiment 1 of the present invention applied to the digital camera described above. This embodiment is an example of RAW data processing. As shown in  FIG. 7 , a process flow thereof includes image development ( 1 ) processing (Step S 101 ), false color (color moire) reduction mode determination processing (Step S 102 ), false color detection processing (Step S 103 ), NR level setting processing (Step S 104 ), NR level determination processing (Step S 105 ), image development ( 2 ) processing (Step S 106 ), image synthesis table creation processing (Step S 107 ), and image synthesis processing (Step S 108 ). Note that the process flow shown in  FIG. 7  is stored in the ROM  108  or the memory card  130 . The above processings will be described in detail below. 
     [Image Development ( 1 ) Processing] 
     In Step S 101 , image development of RAW data is performed. In the image development, an image is generated by performing a series of image processing (WB processing, γ correction processing, interpolation processing, YCbCr conversion processing, color correction processing, edge enhancement processing, and the like) on the RAW data (data on one image). 
     Here, description will be given of noise reduction (NR) not listed in the image processing flow shown in  FIG. 3 . The noise reduction is processing to reduce noise contained in the image. A method for the noise reduction varies depending on when to perform the noise reduction (before or after the interpolation processing or after the YCbCr conversion). With reference to  FIG. 8 , description will be given below of characteristics of each timing of noise reduction and noise reduction methods. 
     (A) Before Interpolation Processing 
     As shown in  FIG. 8A , image data before interpolation processing only has information on one color in each pixel in the same Bayer array as that of the RAW data. 
     (B) after Interpolation Processing 
     By the interpolation processing, lacking color information is interpolated from peripheral pixels. In an example of R information, every pixel has R information as shown in  FIG. 8B . The processing is similarly performed for G and B information, thus allowing each pixel to have information on the three colors, R, G and B. 
     (C) after YCbCr Conversion 
     The RGB signal is converted into a YCbCr signal, and the image data becomes information on luminance (Y) and color difference (CbCr). 
     As an advantage related to the timing of noise reduction, calculation cost is saved because the information amount is smaller before the interpolation processing than after the interpolation processing. The processing of pixels of the same color enables a target pixel to be widely referred to with a small number of peripheral pixels. On the other hand, as a disadvantage, the processing cannot be performed without format conversion because the shot JPEG image is subjected to interpolation processing. 
     The information amount is three times larger after the interpolation processing than before the interpolation processing, and thus the calculation cost is increased. The shot image requires format conversion from YCbCr to RGB. 
     Moreover, while the information of luminance (Y) and color difference (CbCr) is obtained after the YCbCr conversion, false color reduction processing requires only color-difference components, and thus the calculation cost is saved to ⅔ of that after the interpolation processing. Moreover, the shot JPEG image can be processed without format conversion. 
     Considering the above, (A) before the interpolation processing or (C) after the YCbCr conversion described above is suitable as the timing of noise reduction. 
     Here, several noise reduction methods will be presented. 
     (a) Low-Pass Filter (LPF) 
     The low-pass filter is a method of multiplying a target pixel and peripheral pixels by a weight coefficient, respectively, and calculating an average. In the case of processing of the R signal in  FIG. 8A , for example, noise reduction is performed using R 13  as the target pixel and using a 3×3 LPF as shown in  FIG. 9A , thereby obtaining R 13 ′ as follows as a result of the pixel R 13  after the noise reduction.
 
 R 13′=( R 13×4+( R 1+ R 3+ R 5+ R 11+ R 15+ R 21+ R 23+ R 25))/12
 
     In the case where the LPF is used for the color difference shown in  FIG. 8C , Cr 13 ′ as the pixel Cr 13  after the noise reduction is as follows.
 
 Cr 13′=( Cr 13×4+( Cr 7+ Cr 8+ Cr 9+ Cr 12+ Cr 14+ Cr 17+ Cr 18+ Cr 19))/12
 
(b) Median Filter
 
     A median filter is a method of setting an intermediate value among signal values of reference pixels as a signal value of a target pixel. In the case of processing of the R signal in  FIG. 8A , for example, assuming R 13  as the target pixel, the value of R 13  is an intermediate value obtained when changing the order of R 1  to R 25 . 
     (c) ε (Epsilon) Filter 
     A ε filter is a method of setting a certain threshold and calculating an average value of only pixels that meet threshold conditions among reference pixels. In the case of processing of the R signal in  FIG. 8A , for example, a threshold T is set with R 13  being the target pixel, and processing is performed using the ε filter assuming that threshold conditions are met when the signal value is not more than the threshold T. In this case, each of the signal values of R 1  to R 25  is compared with the threshold T, and an average value is calculated only among the signals whose values are not more than the threshold T. The ε filter is characterized by suitable filtering of edges and a flat portion, and thus enables noise reduction to be performed without lowering the resolution while saving the edges. 
     Note that although the noise reduction methods have been listed above, this embodiment is not limited to the above methods as long as false color reduction can be performed. 
     Next, description will be given of a noise reduction level (NR level) indicating the level of noise reduction. 
     (a) In the Case of Using the Low-Pass Filter (LPF) 
     One way of adjusting the NR level of noise reduction using the low-pass filter is using a filter coefficient of the LPF. As a measure of the level of noise reduction, a ratio of a center value ( 4  in  FIG. 9A ) of the LPF to the sum of all coefficients is used. The NR level is increased when the ratio is small, and the NR level is reduced when the ratio is large. In  FIG. 9A , for example, the ratio of the center value of the LPF is 4/12, i.e.; 0.33. When the center value of the LPF is set to 1 as shown in  FIG. 9B , the ratio becomes 1/9, i.e., 0.11, resulting in increased NR level. On the other hand, when the center value of the LPF is set to 8 as shown in  FIG. 9C , the ratio becomes 8/16, i.e., 0.5, resulting in reduced NR level. 
     Another way of adjusting the NR level is a method of changing the number of times of executing the LPF processing. The larger the number of times of executing the LPF processing, the higher the noise reduction level. 
     (b) In the Case of Using the Median Filter 
     One way of adjusting the NR level of noise reduction using the median filter is a method of changing the number of times of executing the median filter processing. As in the case of the LPF, the larger the number of times of executing the median filter processing, the higher the noise reduction level. 
     (c) In the Case of Using the ε (Epsilon) Filter 
     One way of adjusting the NR level of noise reduction using the ε filter is a method of changing threshold setting. 
     Another way of adjusting the NR level is a method of changing the number of times of executing the ε filter processing. The larger the number of times of executing the ε filter processing, the higher the noise reduction level. 
     Note that although the methods for changing the noise reduction level have been listed above, this embodiment is not limited to the above methods as long as the noise reduction level can be changed. 
     The noise reduction level (NR level) during processing in Step S 101  in  FIG. 7  is a noise reduction level (NR 1 ) used for normal shooting regardless of a false color reduction mode. Note that the image development ( 1 ) processing in Step S 101  corresponds to processing by a first image creator. Also, image data created by the image development ( 1 ) processing corresponds to first noise-processed image data. Moreover, Step S 101  serves as a first step. 
     [False Color (Color Moire) Reduction Mode Determination Processing] 
     In Step S 102 , it is determined whether a false color reduction mode is on or off. The false color reduction mode is a mode of performing processing of reducing false colors in a shot image. A camera is equipped with a function to turn on and off the false color reduction mode so that the user can choose before shooting. In Step S 102 , the determination is made using the on/off information on the false color reduction mode previously set by the user. When the false color reduction mode is off, no false color reduction processing is performed, the image created in Step S 101  becomes a final image, and the image is subjected to JPEG compression and recorded in the memory card or the like. On the other hand, when the false color reduction mode is on, the processing advances to Step S 103  for false color reduction processing. 
     [False Color Detection Processing] 
     In Step S 103 , false color detection is performed using the image created in Step S 101 . There have been proposed various false color detection methods. Some examples thereof will be described below. 
     (a) Edge Detection Method 
     Since a false color is likely to be generated in a region with a drastic change in the signal amount, edge extraction is performed on an image to detect the false color. When the image data is YCbCr data after the YCbCr conversion, edge detection is executed by performing filter processing on the image data using an edge extraction filter as shown in  FIG. 10A  for the luminance (Y). In the case of image data shown in  FIG. 11A , for example, the amount of edges to be extracted is as follows.
 
Edge amount= Y 13×8+( Y 7+ Y 8+ Y 9+ Y 12+ Y 14+ Y 17+ Y 18+ Y 19)×(−1)
 
     When the image data is RGB data before the YCbCr conversion, edge extraction is executed by performing filter processing on the image data using an edge extraction filter as shown in  FIG. 10A  for G pixels. In the case of image data shown in  FIG. 11B , for example, the amount of edges to be extracted is as follows.
 
Edge amount= G 13×8+( G 7+ G 8+ G 9+ G 12+ G 14+ G 17+ G 18+ G 19)×(−1)
 
     When the image data is one before the interpolation processing, edge extraction is executed by performing filter processing on the image data using an edge extraction filter as shown in  FIG. 10B  for Gr and Gb pixels. In the case of image data shown in  FIG. 11C , for example, the amount of edges to be extracted is as follows.
 
Edge amount= Gr 13×4+( Gb 7+ Gb 9+ Gb 17+ Gb 19)×(−1)
 
     Considering the horizontal and vertical directions, an edge extraction filter shown in  FIG. 10C  is used.
 
Edge amount= Gr 13×12+( Gr 1+ Gr 3+ Gr 5+ Gb 7+ Gb 9+ Gr 11+ Gr 15+ Gb 17+ Gb 19+ Gr 21+ Gr 23+ Gr 25)×(−1)
 
     Based on a result of the edge extraction, a point where the edge is extracted is regarded as a false color generation spot, and the amount of edges extracted is regarded as a false color generation level. 
     (b) Color Detection Method 
     The brightness, hue and chroma in case of a false color are limited to some extent. Thus, colors included in specific brightness, hue and chroma are detected, and color detection spots and chroma are obtained as a false color detection result. When the image data is the YCbCr data after the YCbCr conversion, luminance (Y) is used as the brightness, and color difference (Cb and Cr) information is used as the hue and chroma. For example, a CbCr space is partitioned by two axes (Axis ( 1 ) and Axis ( 2 )) for the hue as shown in  FIG. 12A , and a hue region between the two axes is narrowed down based on a distance from the origin (corresponding to the chroma), thereby setting a detection area (shaded region). 
       FIG. 12B  shows a case where a luminance Y axis is added to the CbCr space. When the space is narrowed down also for the brightness, a range (Y 1 &lt;Y&lt;Y 2 ) is set in the luminance direction of the CbCr space. Thus, the detection area (the shaded portion in  FIG. 12B ) can be set based on the brightness, hue and chroma. Note that not one detection area but multiple detection areas may be set in the false color generation region. 
     When the image data is the RGB data before the YCbCr conversion, the luminance is calculated, and the luminance Y is used as the brightness and color differences (R-Y and B-Y) are used as the hue and chroma. Alternatively, in a simple manner without calculating the luminance, the G signal is used as the brightness and color differences (R-G and B-G) are used as the hue and chroma. 
     The position of the pixel having information included in the detection area is regarded as the false color generation spot, and the chroma (the distance from the origin in the CbCr space) is regarded as the false color generation level. 
     (c) Combination 
     A method combining the edge detection method and the color detection method enables more accurate detection of false colors. When both of the edge detection and the color detection meet the conditions, false colors are detected. Alternatively, when any of the edge detection and the color detection meets the conditions, false colors are detected. The former case is effective in detecting only a region where a false color is generated, while the latter case has an advantage of detecting all portions suspected to be false colors. 
     The position of the pixel meeting the false color detection conditions is regarded as the false color generation spot. As to the false color generation level, since the two detection methods are used, normalization may be performed using a maximum possible value for the false color generation level of each detection method, and a larger value among the normalized values may be used. Alternatively, a new false color generation level may be calculated by obtaining a weighted average of the false color generation levels. 
     Note that the false color detection of this embodiment is not limited to the methods described above as long as the false color can be detected. 
     [NR Level Setting Processing] 
     In Step S 104 , a NR level (a new NR level: NR 2 ) in image development performed in the next Step S 106  is set based on the result of the false color detection in Step S 103 . As a result of the false color detection, information on the false color generation spot and the false color generation level is obtained. The NR level is set based on the false color generation level. 
     For the NR level setting, a table showing a relationship between the false color generation level and the NR level is previously set. For example, the NR level is set for the range of the false color generation level (FCL) as shown in  FIG. 13 . The larger NR level means stronger NR. When every pixel has information on the false color generation spot and the false color generation level, the NR level is set based on the maximum false color generation level in the entire image. 
     [NR Level Determination Processing] 
     In Step S 105 , the NR level (NR 2 ) set in Step S 104  is compared with the NR level (NR 1 ) obtained in Step S 101 . When the relationship between the two NR levels is NR 1 &lt;NR 2 , false color reduction is achieved by performing stronger NR than the case of Step S 101  in Step S 106 , and thus the processing advances to Step S 106 . On the other hand, when the relationship between the NR levels is NR 1 ≧NR 2 , the NR is weaker than that in Step S 101 , and thus false color reduction is not achieved. Therefore, the image acquired in Step S 101  is outputted as the final image without false color reduction. 
     [Image Development ( 2 ) Processing] 
     In Step S 106 , image development ( 2 ) is performed at the NR level (NR 2 ) determined in Step S 104 . The development is performed from the RAW data by changing only the NR level from Step S 101 . Note that the image development ( 2 ) processing in Step S 106  corresponds to processing by a second image creator. Also, image data created by the image development ( 2 ) processing corresponds to second noise-processed image data. Moreover, Step S 106  serves as a second step. 
     [Image Synthesis Table Creation Processing] 
     In Step S 107 , an image synthesis table is created to determine a synthesis rate for synthesizing the images, which are created in Steps S 101  and S 106 , in Step S 108 . To create the image synthesis table, the result of the false color detection in Step S 103  is used. A method for creating the image synthesis table will be described below by giving a concrete example. 
     Although the information on the false color generation spot and the false color generation level is obtained by the pixel from the false color detection result, the image synthesis table is created by the pixel with reference to the false color generation level for each pixel. 
     A table is previously set, in which the false color generation level and the image synthesis ratio are specified. As shown in  FIG. 14A , for example, an image ratio (synthesis ratio: 0 to 1) of the NR level created in Step S 106  is set for the false color generation level (FCL). The higher the synthesis ratio, the higher the ratio of the image having the NR level (NR 2 ). 
     Also, in the case of an image having a false color generation level as shown in  FIG. 14B , for example, an image synthesis table as shown in  FIG. 14C  is created, in which a synthesis ratio for each pixel is determined based on the false color generation level for each pixel and the synthesis ratio table in  FIG. 14A . 
     Although the image synthesis table is created referring to the image synthesis ratio table for each pixel in  FIG. 14C , the image synthesis table may be created by dividing the image into arbitrary blocks in the horizontal and vertical directions and then calculating an image synthesis ratio from an average of false color generation levels among the blocks. 
     [Image Synthesis Processing] 
     In Step S 108 , the images created in Steps S 101  and S 106  are synthesized according to the image synthesis table created in Step S 107 . 
     In image synthesis, assuming that the coefficient of the image synthesis table is a, the image in Step S 101  is image ( 1 ) and the image in Step S 106  is image ( 2 ), composite pixels are obtained by calculating image ( 1 ) pixels×(1−α)+image ( 2 ) pixels×α for pixels at the same position in image ( 1 ) and image ( 2 ), and a composite image is obtained by calculating the above for all the pixels. Note that the image synthesis table creation processing in Step S 107  and the image synthesis processing in Step S 108  correspond to processing by an image synthesizer. Also, Steps S 107  and S 108  serve as a third step. 
     According to this embodiment, data on two images is created by performing noise reduction processing (the processing of Step S 106  in  FIG. 7 ) with different noise reduction levels (the noise reduction level in Step S 101  in  FIG. 7 ) for one image. Thus, an image not subjected to false color reduction and an image subjected to false color reduction can be created. 
     Since the image synthesis table is creased and the two images are synthesized according to the image synthesis table, the false color reduction level can be easily adjusted. Furthermore, the resolution in the region to adjust the false color reduction level can be easily adjusted by the pixel, by the block or the like, depending on how to create the image synthesis table. 
     Furthermore, the first image and the second image are synthesized by taking a weighted average thereof in accordance with the ratio in the image synthesis table. This makes it possible to change the false color reduction according to the false color generation level in the false color generation portion, and to perform no false color reduction in the other portions. 
     Moreover, as the image synthesis table creation method, the false color reduction in the false color generation spot can be enhanced by increasing the image data ratio with a high noise reduction level in a spot with a high false color generation level. Meanwhile, in a spot with no false color generated, the image data ratio with a low noise reduction level is increased. Thus, a side effect (e.g., color blurring and the like) of strong noise reduction can be minimized. 
     Furthermore, for the false color reduction, it is often the case that special color interpolation processing is adopted or complex processing circuits are mounted. However, in this embodiment, false color reduction can be performed using a general image processing circuit without special false color reduction circuit. 
     Moreover, according to this embodiment, the false color generation level is detected from the first image subjected to the noise reduction processing in Step S 101 , and the noise reduction level is determined for the second image in Step S 104 . Thus, an image can be created, in which more false colors are reduced in the second image than in the first image. In this case, the noise reduction processing is not performed when the conditions are not met. As a result, unnecessary noise reduction processing can be omitted, and thus processing time can be reduced. 
     Embodiment 2 
     Next, Embodiment 2 will be described. Although the false color reduction method using the RAW data has been described in Embodiment 1, it is preferable that false color reduction can be performed also for shot image data. This embodiment is characterized by false color reduction from images stored in JPEG format. 
     As shown in  FIG. 15 , a process flow in this embodiment includes false color (color moire) reduction mode determination processing (Step S 201 ), false color detection processing (Step S 204 ), NR level determination processing (Step S 203 ), NR processing (Step S 204 ), image synthesis table creation processing (Step S 205 ), and image synthesis processing (Step S 206 ). Note that JPEG extension and compression processing is omitted in the process flow shown in  FIG. 15 . 
     The process flow of this embodiment is different from that of Embodiment 1 shown in  FIG. 7  in that noise reduction is further performed on JPEG data already subjected to noise reduction, in the NR level determination processing of Step S 203 . For this reason, it is preferable to set a low NR level, unlike the case of the RAW data. Then, in Step S 204 , only NR processing is performed on the JPEG data. 
     As for the other processing, false color reduction processing is also performed on the JPEG data using the same method as that in Embodiment 1. 
     Note that this embodiment is applicable not only to the JPEG format but also another image format as long as the format has more than one pixel information in one pixel, such as TIFF, PNG and BMP, and does not require information interpolation from peripheral pixels. 
     While the embodiments of the present invention have been described in detail above with reference to the drawings, the above embodiments are merely examples of the present invention. The present invention is not limited to only the configurations of the above embodiments, but, needless to say, includes changes in design and the like without departing from the scope of the present invention. 
     According to the present invention, false colors generated in images can be reduced with a simple process configuration only by creating and synthesizing the first and second noise-processed image data. 
     The present application is based on and claims priority from Japanese Patent Application No. 2012-198043, filed on Sep. 10, 2012, the disclosure of which is hereby incorporated by reference in its entirety.