Camera module, electronic apparatus, and photographing method for image stabilization and increasing resolution

According to one embodiment, a camera module includes an image stabilization section and a superposing section. The image stabilization section adjusts a position of a correction lens such that a second image, which is obtained in succession to a first image, is moved by a movement less than a length of one pixel relative to the first image. The superposing section superposes the first image and one or more of the second images that are taken after the position of the correction lens has been adjusted by the image stabilization section.

CROSS-REFERENCE OF RELATED APPLICATIONS

This application is based upon and claims the benefit of priority from Japanese Patent Application No. 2011-013960, filed on Jan. 26, 2011; the entire contents of all of which are incorporated herein by reference.

FIELD

Embodiments described herein relate generally to a camera module, an electronic apparatus, and a photographing method.

BACKGROUND

Conventionally, there is a technique to take an image with high resolution by an interpolation using successive frames. Such a technique can realize an effective improvement in obtaining high resolution in a movie filming a moving object. Contrary to this, in taking still images, a plurality of frames not including any movement of the object is simply superposed, and merely the resolution is increased by emulation in a filtering process; whereby there has been a case in which defects such as discoloration in an outline portion or the like occur in the image. As aforementioned, according to the conventional technique, there is a problem that there may be a case in which an affected image is taken. Further, especially in a camera module installed in a portable electronic apparatus, due to demands for decreasing size and weight, it is desirable to realize an improvement in obtaining high resolution by a simple configuration.

DETAILED DESCRIPTION

In general, according to one embodiment, a camera module includes an image pickup optical system, an image sensor, an image stabilization section, and a superposing section. A correction lens is assembled in the image pickup optical system. The image sensor takes an image of an object image imaged by the image pickup optical system. The image stabilization section performs an image stabilization by an adjustment of a position of the correction lens. A superposing section superposes a plurality of images obtained by an image-taking by the image sensor. The image stabilization section adjusts the position of the correction lens in an image-taking mode for obtaining higher resolution than resolution corresponding to a pixel number of the image sensor. The image stabilization section adjusts the position of the correction lens such that after a first image is taken by an image-taking by the image sensor, a second image is moved by a movement less than a length of one pixel relative to the first image. The second image is obtained in succession to the first image. The superposing section superposes the first image and one or more of the second images that are taken after the position of the correction lens has been adjusted by the image stabilization section.

Exemplary embodiments of a camera module, an electronic apparatus, and a photographing method will be explained below in detail with reference to the accompanying drawings. The present invention is not limited to the following embodiments.

FIG. 1is a block diagram illustrating a schematic configuration of the electronic apparatus including the camera module of the first embodiment. The electronic apparatus includes a camera module1and an ISP (Image Signal Processor)2. The electronic apparatus is, for example, a digital camera. The electronic apparatus to which the camera module1is adapted may be apparatuses other than the digital camera, for example, a portable terminal with a camera.

The camera module1includes an image pickup optical system10, an image sensor11, an OIS (optical image stabilization)12, a signal processing section13and a frame memory14.

The image pickup optical system10causes the image sensor11to image an object image. The image sensor11converts light taken in from the image pickup optical system10to signal charges, and takes an image of the object image. The OIS12is an optical image stabilization device (image stabilization section) that stabilizes a camera shake. The signal processing section13generates analog image signals by taking in signal values for red (R), green (G), and blue (B) in an order corresponding to a Bayer arrangement, and converts the obtained image signals from an analog format to a digital format. The frame memory14retains images obtained by an image-taking by the image sensor11.

The signal processing section13includes a scaling section15and an superposing section16. The scaling section15performs scaling on an image obtained by the image-taking by the image sensor11. The superposing section16superposes a plurality of images obtained by the image-taking by the image sensor11.

The ISP2includes a camera module I/F20, an image intaking section21, a signal processing section22, and a driver I/F23. RAW images obtained by the image-taking by the camera module1are taken into the image intaking section21from the camera module I/F20.

The signal processing section22performs a signal process on the RAW images taken into the image intaking section21. The driver I/F23outputs the image signals that have undergone the signal process in the signal processing section22to a display driver3. The display driver3displays an image according to the image signals from the driver I/F23.

FIG. 2is a flow chart explaining procedures of a signal process by the signal processing section of the ISP. The signal processing section22(seeFIG. 1) performs shading compensation for the RAW images obtained by the image-taking by the camera module1(step S1). The shading compensation is a compensation of uneven brightness due to a difference in an intensity at a center portion and a periphery portion of the image pickup optical system10(seeFIG. 1).

The signal processing section22performs noise reduction that reduces noise such as a fixed pattern noise, a dark current noise, or a shot noise (step S2) and a resolution reconstruction process (step S3). Next, the signal processing section22performs pixel interpolation (demozaicking) on the digital image signals that are transmitted in the order of the Bayer arrangement (step S4). In the demozaicking, by the interpolation of the image signals obtained by the image-taking of the object image, sensitivity level values of insufficient color components are generated. The signal processing section22composites a color image by demozaicking.

The signal processing section22performs automatic adjustment of white balance (Automatic White Balance Control; AWB) on the color image (step S5). Further, the signal processing section22performs a linear color matrix process for obtaining color reproducibility (step S6) and gamma correction for correcting chroma and brightness of the image to be displayed on a display and the like (step S7).

Note that, the procedures of the signal process illustrated inFIG. 2are a mere example, and other processes may be added thereto, omittable processes may be omitted, and the order may be changed as appropriate. The respective processes may be performed by either of the camera module1and the ISP2, and may be shared therewith.

FIG. 3is a block diagram illustrating a configuration of the OIS. The OIS12includes a correction lens120, a position sensor121, and a position control driver122. The correction lens120is assembled in the image pickup optical system10(seeFIG. 1). The OIS12modifies a focal aim of the image pickup optical system10by an adjustment of a position of the correction lens120.

The position sensor121detects a moving direction and a movement distance of the camera module1by the camera shake. As the position sensor121, a vibrating gyro mechanism may be used, for example. A processor (not illustrated) of the OIS12calculates a direction and a movement by which the correction lens120is to be moved from the detection result of the position sensor121. The position control driver122adjusts the position of the correction lens120according to the direction and the movement calculated by the processor. As the position control driver122, a voice coil motor (VCM) is used, for example.

The OIS12performs image stabilization by adjusting the position of the correction lens120so as to offset the movement of the camera module1due to the camera shake. When the position of the correction lens120is adjusted, the position sensor121performs resetting by a feedback circuit that assumes the current position of the correction lens120to be a zero point.

The camera module1utilizes the OIS12as the device for image stabilization, and in addition thereto, utilizes the OIS12in an image-taking mode for obtaining higher resolution than resolution corresponding to a number of pixels that the image sensor11has.

FIG. 4is a flow chart explaining procedures of a process for obtaining high resolution by the camera module. The camera module1(seeFIG. 1) obtains a first image by the image-taking by the image sensor11(step S11). The frame memory14retains the first image obtained by the camera module1.

The scaling section15performs upscaling of the first image read from the frame memory14(step S12). For example, the scaling section15enlarges the first image by quadruple (doubled to a horizontal direction, and doubled to a vertical direction).

An effect of the upscaling depends on an algorithm used for scaling. The scaling section15can enlarge images without deteriorating image quality by using interpolation of bilinear or bicubic, for example. The scaling section15may perform the upscaling using algorithms other than the bilinear or bicubic.

Next, the OIS12adjusts the position of the correction lens120(step S13). The OIS12adjusts the position of the correction lens120such that a second image, which is obtained in succession to the first image, is moved by a movement less than a length of one pixel relative to the first image.

Assuming a scale factor of the upscaling by the scaling section15to be m (where m is an integer of 2 or more), the movement of the second image relative to the first image with respect to a first direction that is the horizontal direction (x direction) and a second direction that is perpendicular to the first direction and is the vertical direction (y direction) is set to be 1/√m times the length of one pixel after the upscaling. In case of assuming the scale factor of the upscaling to be quadruple (m=4), the movement is ½ the length of one pixel after the upscaling. The camera module1moves the correction lens120such that the second image shifts orthogonally for half a pixel relative to the first image.

In a case where the position sensor121detects movement of the camera module1due to the camera shake upon adjusting the position of the correction lens120, the OIS12performs the image stabilization by deducting the movement for the present step. Due to this, obtaining high resolution and image stabilization can be performed simultaneously.

The camera module1obtains the second image by the image-taking by the image sensor11after having adjusted the position of the correction lens120in step S13(step S14). The frame memory14retains the second image obtained by the camera module1.

The scaling section15performs upscaling of the second image read from the frame memory14(step S15). The scaling section15enlarges the second image by the same scale factor as the upscaling of the first image in step S12.

The superposing section16superposes the first image upscaled in step S12and the second image upscaled in step S15(step S16). The scaling section15performs downscaling on the image obtained by the superposing in step S16(step S17). The scaling section15shrinks the image to the size before the upscaling.

According to the above, the camera module1completes the process for obtaining high resolution. Note that, the camera module1may omit the downscaling in step S17, and may output the image in the size by which the upscaling was performed.

FIG. 5andFIG. 6are conceptual diagrams explaining obtaining high resolution by the superposing of the first image and the second image.FIG. 5illustrates sampling points in the first image P1, sampling points in the second image P2and sampling points in case of superposing the first image P1and the second image P2. All of them have a sensitivity level as a vertical axis, and x coordinate as a horizontal axis.FIG. 6illustrates distribution of the sensitivity level by bar graphs.

The sampling points in the x direction are doubled by shifting the second image P2for half a pixel in the x direction relative to the first image P1. Further, as for the y direction also, the sampling points in the y direction are doubled by shifting the second image P2for half a pixel in the y direction relative to the first image P1. With the sampling points being doubled respectively in the x direction and the y direction, the sampling points are quadrupled in a two-dimensional direction. As aforementioned, even in cases where only several sampling points for the respective images obtained by the image-taking are present, the sampling points can be increased by superposing the images by shifting.

Note that, the camera module1can make the scale factor of the upscaling to be greater than double if such is allowable in view of the processing time and the circuit scale. For example, the scale factor of the upscaling may be sixteen times (quadrupled to the horizontal direction, and quadrupled to the vertical direction, m=16), and the movement of the second image relative to the first image may be ¼ of the length of one pixel. In this case, three pieces of the second image that have been shifted by ¼ of the length of one pixel are obtained, and by superposing them with the first image, the sampling points for each of the images can be increased by sixteen times. The superposing section16is not limited to superposing one second image to the first image, and it may superpose a plurality of second images.

By employing techniques to increase the sampling points by superposing the images that are shifted relative to each other, the camera module1can suppress occurrences of defects such as discoloration and the like than in case of performing filtering process for spuriously obtaining high resolution. Further, by diverting the OIS12installed for image stabilization to the process for obtaining high resolution, the camera module1can realize obtaining high resolution by a simple configuration compared to adding constituent components separately. Due to this, the camera module1can obtain images with high resolution and high quality by the simple configuration.

FIG. 7is a block diagram illustrating a schematic configuration of an electronic apparatus including a camera module of the second embodiment. In the second embodiment, components identical to those of the first embodiment will be given the same reference signs, and an overlapping explanation thereof will not be repeated. The signal processing section13includes the scaling section15, the superposing section16, and a resolution reconstruction section17. The resolution reconstruction section17performs resolution reconstruction on an image obtained by an image-taking by the image sensor11.

The resolution reconstruction section17performs the resolution reconstruction by estimating a lens character comprised by the image pickup optical system10such as scale factor chromatic aberration and amount of blur that cause blurs of color in a contour. The resolution reconstruction section17estimates the lens character comprised by the image pickup optical system10by referencing parameters stored in advance. As the lens character, a PSF (Point Spread Function) that is an optical transmission coefficient is used, for example. The resolution reconstruction section17estimates the PSF by using a least squares method for example.

The parameters referenced by the resolution reconstruction section17in estimating the lens character are, for example, ones indicating a manufacturing error of components such as a lens and the like, an assembly error of the components, an attachment error of the camera module1and the like. Due to this, the effect of the resolution reconstruction is securely obtained. The parameters are stored in, for example, an OTP (One Time Programmable memory, not illustrated) provided in the camera module1, or the ISP2.

The effect of the resolution reconstruction depends on an algorithm used for the reconstruction. As the process of the resolution reconstruction, in order to reconstruct an image similar to the original object image, a Richardson-Lucy method is used, for example.

FIG. 8is a flow chart explaining procedures of a process for obtaining high resolution by the camera module. The camera module1obtains the first image by the image-taking by the image sensor11(step S21). The frame memory14retains the first image obtained by the camera module1.

The resolution reconstruction section17performs resolution reconstruction of the first image read from the frame memory14(step S22). The scaling section15performs upscaling of the first image that has undergone the resolution reconstruction by the resolution reconstruction section17(step S23).

Next, the OIS12adjusts the position of the correction lens120(step S24). The OIS12adjusts the position of the correction lens120such that a second image, which is obtained in succession to the first image, is moved by a movement less than a length of one pixel relative to the first image.

The camera module1obtains the second image by the image-taking by the image sensor11after having adjusted the position of the correction lens120in step S24(step S25). The frame memory14retains the second image obtained by the camera module1.

The resolution reconstruction section17performs resolution reconstruction of the second image read from the frame memory14(step S26). The scaling section15performs upscaling of the second image that has undergone the resolution reconstruction by the resolution reconstruction section17(step S27). The scaling section15enlarges the second image by the same scale factor as the upscaling of the first image in step S23.

The superposing section16superposes the first image upscaled in step S23and the second image upscaled in step S27(step S28). The scaling section15performs downscaling on the image obtained by the superposing in step S28(step S29). The scaling section15shrinks the image to the size before the upscaling.

According to the above, the camera module1completes the process for obtaining high resolution. Note that, the camera module1may omit the downscaling in step S29, and may output the image in the size by which the upscaling was performed.

Similar to the first embodiment, the second embodiment can obtain an image with high resolution and high quality by a simple configuration. Further, the second embodiment can obtain further quality in resolution by superposing the first image and the second image that have undergone the resolution reconstruction.