Abstract:
An image processing device has an image sensing device, an optical black (OB) level detection unit that measures an OB level from an OB area in an image signal output by the image sensing device, an OB correction unit that corrects an OB level of an image signal output after the OB area, and a mode changeover unit that controls switching between a first mode that outputs an image signal of predetermined pixels and a second mode that outputs an image signal from fewer pixels than the predetermined pixels. The OB level detection unit measures the OB level from an image signal output during the second mode, and the OB level correction unit calculates an optical black level for the first mode from the measured optical black level, based on image sensing conditions to be used in the first mode, and corrects the optical black level during the first mode.

Description:
FIELD OF THE INVENTION  
       [0001]     The present invention relates to an image signal processing apparatus and image signal processing method capable of performing optical black correction.  
       BACKGROUND OF THE INVENTION  
       [0002]     In an ordinary image sensing apparatus having a lens system and an image sensing element, there is a risk of such obstacles as, for example, shading of an image to be sensed due to a drop in marginal lumination caused by the lens system. To counteract this effect, the lens system is designed using multiple lenses, for example, to prevent the occurrence of such obstacles. However, such multi-lens lens systems are expensive, and moreover, in most cases are difficult to use in compact cameras and the like.  
         [0003]     By contrast, where signal intake is carried out according to XY coordinates as with an instrument that uses a semiconductor image sensing element, for example, the image can be corrected by digital processing of the image signal. Consequently, conventionally, techniques of digitally correcting distortion due to image sensing with an inexpensive lens system, or of such lens shading as a drop in marginal lumination and color migration, are proposed. For example, in JP-A-2004-266750, a correction technique like the following is proposed: In general, it can be thought that the lens shading is a function of the distance from the lens system optical axis to correction pixels. Therefore, in order to correct lens shading, first, a distance d from the lens system optical axis to the pixel to be corrected is calculated using Pythagoras&#39;s theorem d=√{square root over (x 2 +y 2 )}. Then, a shading coefficient that is a function of the distance from the lens system optical axis to the pixel to be corrected is applied to each of the pixels to be corrected, thereby implementing the correction.  
         [0004]     When applying this type of shading correction, usually noise caused by dark current and the like of the pixels is corrected first, so as not to over-correct the noise by shading correction.  
         [0005]     In addition, in a white point detection process for the purpose of WB (white balance) correction as well, detection cannot be done correctly if noise is present, and therefor, usually, the pixel noise is corrected first.  
         [0006]     As a result, a method that corrects noise by measuring the OB level and subtracting the OB level from the values of the pixels (hereinafter called “OB correction”) is taken as the usual method of correcting noise. With this method, in a monitor mode that continuously reads field images while sampling pixels from the image sensing element for confirming the subject and processes the image signal (an electronic viewfinder display mode; hereinafter referred to as “EVF display mode”), the OB level measurement results of the immediately preceding field image are used to correct the OB level difference of the current field image as shown in  FIG. 13 . In addition, in a “still image sensing mode (main image)” that reads all the pixels from the image sensing element in single shot or a continuous shot for recording an image or images and processes the image signals, first, the OB level is measured when the image signal is read from the image sensing element as shown in  FIG. 13 . Then, the image signal is temporarily held in memory, and the OB level is subtracted when the image signal is read from the memory.  
         [0007]     However, when the image signal is temporarily held in the memory and then read again for OB correction in the still image sensing mode as described above, the interval to the next image sensing lengthens, leading to user stress and lost image-sensing opportunities as a result.  
       SUMMARY OF THE INVENTION  
       [0008]     The present invention has been made in light of the above-described situation, and has as its object to make it possible to carry out optical black correction without temporarily storing the image signal in the memory, shortening the interval to the next image sensing.  
         [0009]     According to the present invention, the foregoing object is attained by providing an image signal processing apparatus comprising:  
         [0010]     an image sensing device that outputs an image signal of an object;  
         [0011]     an optical black level detection unit that measures an optical black level from an image signal from an optical black area in the image signal output by the image sensing device;  
         [0012]     an optical black correction unit that corrects an optical black level of the image signal output from the image sensing device after the image signal from the optical black area is output, based on the optical black level measured by the optical black level detection unit; and  
         [0013]     a mode changeover unit that controls switching between a first mode that outputs an image signal of predetermined pixels of the image sensing device and a second mode that outputs an image signal from fewer pixels than the predetermined pixels of the image sensing device,  
         [0014]     wherein the optical black level detection unit measures the optical black level from the image signal output from the image sensing device during the second mode, and the optical black correction unit calculates an optical black level for the first mode from the optical black level measured by the optical black level detection unit, based on image sensing conditions to be used in the first mode, and corrects the optical black level of the image signal output by the image sensing device during the first mode based on the calculated optical black level for the first mode.  
         [0015]     According to the present invention, the foregoing object is also attained by providing an image signal processing apparatus comprising:  
         [0016]     an image sensing device that outputs an image signal of an object;  
         [0017]     an optical black level detection unit that measures an optical black level from an image signal from an optical black area in the image signal output by the image sensing device;  
         [0018]     an optical black correction unit that corrects an optical black level of the image signal output from the image sensing device after the image signal from the optical black area is output, based on the optical black level measured by the optical black level detection unit; and  
         [0019]     a mode changeover unit that controls switching between a first mode that outputs an image signal of predetermined pixels of the image sensing device and a second mode that outputs an image signal from fewer pixels than the predetermined pixels of the image sensing device,  
         [0020]     wherein the optical black level detection unit measures the optical black level from the image signal output from the image sensing device during the second mode under image sensing conditions to be used in the first mode, and the optical black correction unit corrects the optical black level of the image signal output by the image sensing device during the first mode based on the measured optical black level.  
         [0021]     According to the present invention, the foregoing object is also attained by providing an image signal processing method for processing an image signal of an image of an object output from an image sensing device, comprising:  
         [0022]     receiving a mode changeover instruction to switch between a first mode that outputs an image signal of predetermined pixels of the image sensing device and a second mode that outputs an image signal from fewer pixels than the predetermined pixels of the image sensing device;  
         [0023]     measuring an optical black level from an image signal from an optical black area in the image signal output by the image sensing device in the second mode prior to switching from the second mode to the first mode;  
         [0024]     calculating the optical black level for the first mode from the measured optical black level based on image sensing conditions to be used in the first mode; and  
         [0025]     correcting the calculated optical black level for an image signal obtained from the image sensing device in the first mode after switching from the second mode to the first mode.  
         [0026]     According to the present invention, the foregoing object is also attained by providing an image signal processing method for processing an image signal of an image of an object output from an image sensing device, comprising:  
         [0027]     receiving a mode changeover instruction to switch between a first mode that outputs an image signal of predetermined pixels of the image sensing device and a second mode that outputs an image signal from fewer pixels than the predetermined pixels of the image sensing device;  
         [0028]     measuring an optical black level from an image signal from an optical black area in the image signal output by the image sensing device during the second mode under image sensing conditions to be used in the first mode prior to switching from the second mode to the first mode; and  
         [0029]     correcting, based on the measured optical black level, an optical black level of an image signal output by the image sensing device in the first mode after switching from the second mode to the first mode.  
         [0030]     Other features and advantages of the present invention will be apparent from the following description when taken in conjunction with the accompanying drawings, in which like reference characters designate the same or similar parts throughout the figures thereof. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0031]     The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.  
         [0032]      FIG. 1  is a schematic block diagram illustrating an image signal processing apparatus according to an embodiment of the present invention;  
         [0033]      FIG. 2  is a flow chart illustrating the operation of a system control circuit of an image signal processing apparatus according to a first embodiment of the present invention;  
         [0034]      FIG. 3  is a schematic diagram for illustrating the sensor reading operation of the image signal processing apparatus according to the first embodiment of the present invention;  
         [0035]      FIG. 4  is a timing chart illustrating the operation of the image signal processing apparatus according to the first embodiment of the present invention;  
         [0036]      FIG. 5  is a schematic diagram for illustrating the sensor reading operation of the image signal processing apparatus according to the first embodiment of the present invention;  
         [0037]      FIG. 6  is a timing chart illustrating the operation of the image signal processing apparatus according to the first embodiment of the present invention;  
         [0038]      FIG. 7  is a flow chart illustrating the operation of an image signal processing apparatus according to a second embodiment of the present invention;  
         [0039]      FIG. 8  is a schematic diagram for illustrating the sensor reading operation of the image signal processing apparatus according to the second embodiment of the present invention;  
         [0040]      FIG. 9  is a timing chart illustrating the operation of the image signal processing apparatus according to the second embodiment of the present invention;  
         [0041]      FIG. 10  is a flow chart illustrating the operation of a system control circuit of the image signal processing apparatus according to another embodiment of the present invention;  
         [0042]      FIG. 11  is a schematic diagram for illustrating the sensor reading operation of an image signal processing apparatus according to another embodiment of the present invention;  
         [0043]      FIG. 12  is a timing chart illustrating the operation of an image signal processing apparatus according to another embodiment of the present invention; and  
         [0044]      FIG. 13  is a schematic diagram for illustrating the sensor reading operation of an image signal processing apparatus of the conventional art. 
     
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS  
       [0045]     Preferred embodiments of the present invention will now be described in detail in accordance with the accompanying drawings.  
       First Embodiment  
       [0046]      FIG. 1  is a schematic block diagram illustrating an image signal processing apparatus according to an embodiment of the present invention. In  FIG. 1 , reference numeral  100  designates an image signal processing apparatus. It should be noted that an electronic camera or the like can be given as one example of an image signal processing apparatus.  
         [0047]     Reference numeral  10  designates an image sensing lens that optically focuses an image to be sensed,  11  designates a shutter that adjusts exposure of the image to be sensed,  12  designates an image sensing element that converts the image into an analog electrical signal, and  14  designates an A/D converter that converts the analog signal output of the image sensing element  12  into a digital signal. Here, the digital data output from the A/D converter  14  is hereinafter called CCD-RAW data.  
         [0048]     Reference numeral  16  designates a D/A converter and  18  designates an image display unit composed of a TFT LCD or the like. Image data for display written to a memory  40  is converted from digital data to analog data through the D/A converter  16  and displayed by the image display unit  18 . In addition, continuously reading image signals from the image sensing element  12  and successively displaying the signals on the image display unit  18  provides an electronic viewfinder capability.  
         [0049]     Reference numeral  20  designates a recording medium such as a memory card or a hard disk, on which sensed image data and the like is recorded.  
         [0050]     Reference numeral  30  designates an image processing circuit, which performs WB correction and such developmental processing as predetermined pixel interpolation, color conversion, and resizing on CCD-RAW data to be sensed or on recorded CCD-RAW data.  
         [0051]     Reference numeral  40  designates the memory for developing data from a ROM  96  and for temporarily storing sensed image data, provided with sufficient storage capacity to store a predetermined number of frames of still images or a predetermined length of time of moving images. For example, digital data of an image output from the A/D-converter  14  is written into the memory  40  via a memory control circuit  50 , the image processing circuit  30  and a JPEG circuit  60  or directly through the memory control circuit  50 .  
         [0052]     Reference numeral  50  designates the memory control circuit, which controls the data flow to the A/D converter  14 , the D/A converter  16 , the recording medium  20 , the image processing circuit  30 , the memory  40 , the JPEG circuit  60 , an OB detection circuit  70 , an OB correction circuit  72 , a shading correction circuit  74  and a WB detection circuit  76 .  
         [0053]     Reference numeral  60  designates the JPEG circuit that compresses and expands the image data using baseline JPEG.  
         [0054]     Reference numeral  70  designates the OB detection circuit that measures the OB (optical black) level.  
         [0055]     The OB detection circuit  70  measures the OB level for each of R (red), G (green) and B (blue) colors, for instance, depending on the color filter arrangement of the image sensing element  12 .  
         [0056]     Reference numeral  72  designates the OB correction circuit that performs OB correction by clamping the OB level measured by the OB detection circuit  70  from the image signal.  
         [0057]     Reference numeral  74  designates the shading correction circuit, which multiplies the image signal by a shading correction-coefficient corresponding to the distance from the optical axis. In addition, the present invention is not limited to such an arrangement, and alternatively, for example, the image signal processing apparatus may be constructed so that the image signal is multiplied by a shading correction coefficient corresponding to the horizontal coordinate or by a shading correction coefficient corresponding to the vertical coordinate, or multiplied by shading correction coefficients corresponding to two-dimensional coordinates.  
         [0058]     Reference numeral  76  designates the WB (white balance) control circuit. Reference numeral  90  indicates a system control circuit that carries out operating control of the image sensing apparatus  100  as a whole and of the circuits that comprise the image sensing apparatus  100 , according to either the settings of, e.g., a mode dial switch of a control unit  92  or the contents stored in the ROM  96  (read-only memory).  
         [0059]     Reference numeral  92  designates the control unit, capable of switching and setting functional modes, for example, power off, image sensing mode, playback mode and the like. Reference numeral  96  designates the read-only memory (ROM), in which programs used by the system control circuit  50  and shading correction coefficients used by the shading correction circuit  74  are stored in advance.  
         [0060]     A description will now be given of the operation of an image signal processing apparatus of a first embodiment of the present invention using  FIGS. 2-6 .  
         [0061]      FIG. 3  illustrates an exposed area and an OB area of a signal read from the image sensing element  12 , and a shift from an EVF display (EVF reading) to a main image reading. With EVF reading, the apparatus is driven so that an image signal from relatively few pixels selected as suitable for the image display unit  18  are read. With main image reading, the apparatus is driven so that an image signal is read from all or comparatively many of the pixels of the image sensing element  12 . In the diagram, a rectangle  301  is the exposed area and an area  302  of slanted lines around the rectangle  301  is the OB area. VD is a vertical sync timing signal. The OB level of the main image is measured from the area of dashed-line rectangle  303  by the OB detection circuit  70 .  
         [0062]      FIG. 4  is a timing chart illustrating the process shown in  FIG. 3 . When T is between t 0  and t 1 , the apparatus performs EVF display through an EVF display process. When T is between t 1  and t 2 , the apparatus exposes the main image. When T is between t 2  and t 3 , the apparatus reads the main image. At this time, the OB detection circuit  70  measures the OB level from the vertical OB area shown as the dashed-line rectangle  303  shown in  FIG. 3 , and performs OB correction on the image signal read thereafter with the OB correction circuit  72 . The OB-corrected image signal is shading corrected by the shading correction circuit  74 . In addition, the apparatus measures the WB correction value from the OB level difference-corrected image signal with the WB detection circuit  76  and carries out WB correction.  
         [0063]     Although the method of utilizing the vertical OB area of the main image is described as the method of measuring the OB level, it should be noted that the present invention is not limited thereto. For example, as shown in  FIGS. 5 and 6 , OB correction may be implemented by measuring the OB level at each line from the horizontal OB area of the main image.  
         [0064]      FIG. 2  is a flow chart illustrating the control exerted by the system control circuit  90  on the processes shown in  FIGS. 3 and 4 .  
         [0065]     If during operation in the EVF display mode (step S 10 ) the release button is pressed by the user (step S 12 ), main image sensing is begun. The main image is exposed (step S 14 ), the shutter is closed at a predetermined shutter speed (step S 16 ), and the image signal of an optical image of an object formed on the image sensing element  12  is read (step S 18 ). OB level measurement, OB correction, shading correction and WB correction are performed on the read image signal by the procedures explained with reference to FIGS.  3  to  6  (step S 20 ).  
       Second Embodiment  
       [0066]     A schematic block diagram for illustrating the image signal processing apparatus of a second embodiment of the present invention is the same as  FIG. 1 . A description is now given of the operation of the image signal processing apparatus of the second embodiment of the present invention, using FIGS.  7  to  12 .  
         [0067]      FIG. 8  is a diagram showing the exposed area and the OB area of a signal read from the image sensing element  12 , and the shift from EVF reading to main image sensing. In the diagram, a rectangle  801  is the image signal area and an area  802  of slanted lines around the rectangle  801  is the OB area. VD is a vertical sync timing signal. The OB level of the main image is measured from the area of dashed-line rectangle  803  by the OB detection circuit  70 .  
         [0068]      FIG. 9  is a timing chart for illustrating the process shown in  FIG. 8 . At T between t 0  and t 1 , EVF display is performed by an EVF display process. In addition, the OB detection circuit  70  measures the OB level from the OB area shown as the dashed-line rectangle  803  shown in  FIG. 8 . Even during operation, the OB level is measured from the immediately preceding field and OB correction is carried out with that measured OB level, but measurement of the OB level is carried out using the OB area signal prior to OB correction.  
         [0069]     At T between t 1  and t 2 , the main image is exposed. At this time, the OB level of the main image is calculated from the OB level measurement obtained by the immediately preceding EVF reading using such parameters as the shutter speed and sensitivity of the EVF display process of the immediately preceding field, as well as the shutter speed and sensitivity during main image exposure. At T between t 2  and t 3 , the main image is read. The read image signal is OB corrected by the OB correction circuit  72  using the calculated OB level. The OB-corrected image signal is shading corrected by the shading correction circuit  74 . In addition, a WB correction value is measured from the OB-corrected OB image signal by the WB detection circuit  76  and WB correction is carried out.  
         [0070]      FIG. 7  is a flow chart for illustrating the control exerted by the system control circuit  90  over the processes shown in  FIGS. 8 and 9 .  
         [0071]     If during operation in the EVF display mode (step S 100 ) the release button is pressed by the user (step S 102 ), main image sensing is begun. As the main image is exposed, the OB level of the main image is calculated from OB measurement of the EVF display image (step S 104 ). Thereafter, the shutter is closed at a predetermined shutter speed (step S 106 ) and the image signal focused on the image sensing element  12  is read (step S 108 ). OB correction, shading correction and WB correction are performed on the read image signal by the procedures illustrated in  FIGS. 8-9  (step S 110 ).  
         [0072]     It should be noted that the present invention is not limited to the method described above, and alternatively, as another embodiment, the OB level of the main image may be obtained in the manner described below.  
         [0073]      FIG. 11  is a diagram showing the exposed area and the OB area of a signal read from the image sensing element  12 , and the shift from EVF display to main image sensing. In the diagram, a rectangle  1101  is the image signal area and an area  1102  of slanted lines around the rectangle  1101  is the OB area. VD is a vertical sync timing signal. The OB level for the main image is measured from the area of dashed-line rectangle  1103  by the OB detection circuit  70 .  
         [0074]      FIG. 12  is a timing chart for illustrating the process shown in  FIG. 11 . Up to T=t 0 , EVF display is performed by an EVF display process. At T between t 0  and t 1 , the shutter speed, sensitivity and other parameters are matched to main image processes and EVF reading is carried out, and the OB level is measured by the OB detection circuit  70  from the area indicated by the dashed-line rectangle shown in  FIG. 11  but without EVF display. Even during EVF display, the OB level is measured from the immediately preceding EVF display image and OB correction is carried out with that measured OB level, but measurement of the OB level is carried out using the OB area signal prior to OB correction.  
         [0075]     At T between t 1  and t 2 , the main image is exposed. At T between t 2  and t 3 , the main image is read. The read image signal is OB corrected by the OB correction circuit  72  using the OB level measured in the immediately preceding EVF display process. The OB-corrected image signal is shading corrected by the shading correction circuit  74 . In addition, a WB correction value is measured from the OB-corrected image signal by the WB detection circuit  76  and WB correction is carried out.  
         [0076]      FIG. 10  is a flow chart for illustrating the control exerted by the system control circuit  90  on the processes shown in  FIGS. 11 and 12 .  
         [0077]     If during operation in the EVF display mode (step S 200 ) the release button is pressed by the user (step S 202 ), the shutter speed, sensitivity and other parameters of EVF display operation are matched to those to be used in the main image sensing and the OB level measured (step S 204 ). Then, main image sensing is begun. The main image is exposed (step S 206 ), the shutter is closed at a predetermined shutter speed (step S 208 ), and the image signal of an optical image of an object formed on the image sensing element  12  is read (step S 210 ). OB correction, shading correction and WB measurement are performed on the read image signal by the procedures illustrated in  FIGS. 11-12  (step S 212 ).  
         [0078]     As described above, according to the second embodiment of the present invention, OB correction is implemented during reading of the main image from the sensor, which allows OB correction of a main image to overlap with such processes as luminance shading correction, WB correction value measurement and the like, thus enabling the interval to the next image sensing to be shortened.  
         [0079]     It should be noted that, in the second embodiment, in the event that the OB level cannot be measured in the EVF display mode, control may be arranged so as to measure the OB level from the main image as described with respect to the first embodiment.  
       Other Embodiments  
       [0080]     Furthermore, the invention can be implemented by supplying a software program, which implements the functions of the foregoing embodiments, directly or indirectly, to a system or apparatus, reading the supplied program code with a computer of the system or apparatus, and then executing the program code. In this case, so long as the system or apparatus has the functions of the program, the mode of implementation need not rely upon a program.  
         [0081]     Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the claims of the present invention also cover a computer program for the purpose of implementing the functions of the present invention.  
         [0082]     In this case, so long as the system or apparatus has the functions of the program, the program may be executed in any form, such as an object code, a program executed by an interpreter, or scrip data supplied to an operating system.  
         [0083]     Examples of storage medium that can be used for supplying the program are a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a CD-RW, a magnetic tape, a non-volatile type memory card, a ROM, and a DVD (DVD-ROM and a DVD-R).  
         [0084]     As for the method of supplying the program, a client computer can be connected to a website on the Internet using a browser of the client computer, and the computer program of the present invention or an automatically-installable compressed file of the program can be downloaded to a recording medium such as a hard disk. Further, the program of the present invention can be supplied by dividing the program code constituting the program into a plurality of files and downloading the files from different websites. In other words, a WWW (World Wide Web) server that downloads, to multiple users, the program files that implement the functions of the present invention by computer is also covered by the claims of the present invention.  
         [0085]     It is also possible to encrypt and store the program of the present invention on a storage medium such as a CD-ROM, distribute the storage medium to users, allow users who meet certain requirements to download decryption key information from a website via the Internet, and allow these users to decrypt the encrypted program by using the key information, whereby the program is installed in the user computer.  
         [0086]     Besides the cases in which the aforementioned functions according to the embodiments are implemented by executing the read program by computer, an operating system or the like running on the computer may perform all or a part of the actual processing so that the functions of the foregoing embodiments can be implemented by this processing.  
         [0087]     Furthermore, after the program read from the storage medium is written to a function expansion board inserted into the computer or to a memory provided in a function expansion unit connected to the computer, a CPU or the like mounted on the function expansion board or function expansion unit performs all or a part of the actual processing so that the functions of the foregoing embodiments can be implemented by this processing.  
         [0088]     As many apparently widely different embodiments of the present invention can be made without departing from the spirit and scope thereof, it is to be understood that the invention is not limited to the specific embodiments thereof except as defined in the appended claims.  
         [0089]     This application claims the benefit of Japanese Patent Application No. 2005-144344 filed on May 17, 2005, which is hereby incorporated by reference herein in its entirety.