Imaging device and method of obtaining image

An imaging device including an image sensor, includes an image obtaining control unit that obtains plural continuous images of an object taken by the image sensor; a size determining unit that determines an image size of reduced images based on photographing information of at least one of the images, and generates reduced images from the images obtained by the image obtaining control unit based on the determined image size; a motion vector detection unit that detects a motion vector of the reduced images; and an image synthesis unit that synthesizes the images obtained by the image obtaining control unit based on the motion vector detected by the motion vector detection unit to obtain a synthesized image.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an imaging device and a method of obtaining an image and more specifically, to an imaging device and a method of obtaining an image, capable of obtaining an image appropriately corrected by electronic image stabilization.

2. Description of the Related Art

Conventionally, a technique of image stabilization for an imaging device such as a digital camera or the like, includes an optical image stabilization technique in which a position of an imaging surface or a lens with respect to the imaging surface is shifted to a direction opposite to the direction of hand movement or the like, or an electronic image stabilization technique in which a high-speed shutter is used to obtain plural time-division images, a specific point of an object in the images is used as a reference to synthesize the images to obtain a synthesized image. By using the electronic image stabilization technique, a mechanical structure is not necessary added since it is not necessary to change the position of the imaging surface or the lens to correct the hand movement, so that there is merit that a space and a cost of the imaging device can be reduced.

For the electronic image stabilization technique, it is disclosed in Patent Document 1, for example, that plural images are obtained, a motion vector is detected based on the obtained images by using one of the images as a reference, and a synthesized image is generated.

Further, for the electronic image stabilization technique, a corrected image is obtained by detecting a motion vector based on plural obtained images and synthesizing the images based on the detected motion vector. When detecting the motion vector, the obtained images are reduced to images having a predetermined size where the size of the reduced images is set at a predetermined value.

However, there may be a case where a motion vector with a higher accuracy is required, or a case where a faster speed is required although a precise motion vector is not as necessary, based on a condition in obtaining an image or an image.

It is not disclosed in Patent Document 1 about the image size of images for detecting a motion vector.

PATENT DOCUMENT

SUMMARY OF THE INVENTION

The present invention is made in light of the above problems, and provides an imaging device and a method of obtaining an image, capable of obtaining an image appropriately corrected by electronic image stabilization.

According to an embodiment, there is provided an imaging device including an image sensor, further including an image obtaining control unit that obtains plural continuous images of an object taken by the image sensor; a size determining unit that determines an image size of reduced images based on photographing information of at least one of the images, and generates the reduced images from the images obtained by the image obtaining control unit based on the determined image size; a motion vector detection unit that detects a motion vector of the reduced images; and an image synthesis unit that synthesizes the images obtained by the image obtaining control unit based on the motion vector detected by the motion vector detection unit to obtain a synthesized image.

According to another embodiment, there is provided a method of obtaining an image by an image sensor, including, obtaining plural continuous images of an object taken by the image sensor; determining an image size of reduced images based on photographing information of at least one of the images; generating the reduced images from the obtained images based on the determined image size; detecting a motion vector of the reduced images; and synthesizing the obtained images based on the detected motion vector to obtain a synthesized image.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

It is to be noted that, in the explanation of the drawings, the same components are given the same reference numerals, and explanations are not repeated.

According to the embodiment, when correcting an image by electronic image stabilization, the image size of images used for detecting a motion vector is determined based on photographing information such as setting information of an imaging device in obtaining an image, or characteristic information of an obtained image. Here, the photographing information may be capable of defining a level of accuracy necessary for the motion vector (hereinafter simply referred to as “necessary accuracy”. Therefore, the size of the images used for detecting a motion vector is determined in accordance with the necessary accuracy of the photographing information such that the higher the necessary accuracy, the larger the image size becomes. With this structure, the motion vector is appropriately detected in regard to accuracy and speed.

In other words, according to the embodiment, by selecting a higher accuracy for detecting a motion vector or a faster speed for detecting a motion vector, the motion vector is appropriately detected to obtain appropriate image stabilization.

FIG. 1Ais a top view,FIG. 1Bis an elevation view, andFIG. 1Cis a back elevation view, of an example of an imaging device1. Although the imaging device1of this embodiment is shown as a digital camera, the structure and operation of the imaging device is not limited to that of the digital camera.

The imaging device1includes a sub liquid crystal display (LCD)11, a memory card-battery unit12, a flash lamp13, an optical viewfinder14, a ranging unit15, a remote control light receiver16, a lens-barrel unit17, an LED for automatic focusing (AF)18, an LED for flash19, an LCD monitor20, and switches SW1to SW13.

FIG. 2A to 2Care block diagrams showing an example of a system structure of the imaging device1of the embodiment.

The imaging device1further includes a Charge Coupled Device (CCD)31, a front-end integrated circuit (F/E-IC)32, a Synchronous Dynamic Random Access Memory (SDRAM)33, a digital camera processor (hereinafter simply referred to as a “processor”)34, a Random Access Memory (RAM)35, a built-in memory36, a Read-Only Memory (ROM)37, a voice input unit38, an audio play unit39, a flash lamp circuit40(just shown as “CIRCUIT”, an LCD driver41, a sub Central Processing Unit (CPU)42, an operational key unit43, a buzzer44, an acceleration sensor45, an Universal Serial Bus (USB) connector46, a serial driver circuit47, an RS-232C connector48, an LCD driver49, a video amplifier50, a video jack51, a memory card slot52, and a memory card53, in addition to the sub LCD11, the flash lamp13, the ranging unit15, the remote control light receiver16, the lens-barrel unit17, the LED for AF18, the LED for flash19, the LCD monitor20, which are also shown inFIG. 1AtoFIG. 1C.

The lens-barrel unit17includes a zooming optical system171composed of a zoom lens171aand a zoom motor171b, a focusing optical system172composed of a focus lens172aand a focus motor172b, an aperture unit173composed of an aperture173aand an aperture motor173b, a mechanical shutter unit174composed of a mechanical shutter174aand a mechanical shutter motor174b, and a motor driver175.

The F/E-IC32includes a Correlated Double Sampling (CDS) circuit321, an Automatic Gain Control (AGC) circuit322, an analog digital converter (A/D)323, and a timing generator (TG)324.

The CDS circuit321performs a correlated double sampling in order to remove noise in an image (an image signal). The AGC circuit322performs an automatic gain control. TG324generates a driving timing signal based on a vertical driving pulse VD and a horizontal driving pulse HD.

The processor34includes an Inter Integrated Circuit (I2C) block341, a CCD1signal processing block342, a CCD2signal processing block343, a CPU block344, a local Static Random Access Memory (SRAM)345, an USB block346, a serial block347, a JPEG codec block348, a resize block349, a TV signal display block350, and a memory card controller block351. These blocks341to351are connected with each other via a BUS-line (not shown in the drawings).

The voice input unit38includes a voice recording circuit (just shown as “CIRCUIT”)381, a microphone amplifier (just shown as “AMP”)382, and a microphone383. The audio play unit39includes an audio play circuit (just shown as “CIRCUIT”)391, an audio amplifier (just shown as “AMP”)392, and a speaker393.

As described above, the imaging device1of the embodiment has a function of a digital camera. Specifically, as shown inFIG. 1A, the switches SW1and SW2provided at an upper part of the imaging device are for release and mode dial, respectively.

As shown inFIG. 1B, a cover is further provided at a side part of the imaging device1to cover the memory card-battery unit12in which the memory card53and a battery are installed. Images taken by the imaging device1may be stored in the memory card53. The components of the imaging device1are driven by the battery.

Further as shown inFIG. 1B, the flash lamp13, the optical viewfinder14, the ranging unit15, the remote control light receiver16, and the lens-barrel unit17are provided at a front surface of the imaging device1.

The flash lamp13provides light when obtaining an image. The optical viewfinder14helps a user to find a target or an object through the optical lenses such as the zoom lens171aand the focus lens172a. The remote control light receiver16receives a remote control signal such as infrared rays or the like sent from an external remote control device.

As shown inFIG. 1C, the optical viewfinder14, the LED for AF18, the LED for flash19, the LCD monitor20, the switch SW3for wide-angle zoom (WIDE), the switch SW4for telephoto zoom (TELE), the switch SW5for displaying an obtained image or video, the switch SW6for self-timer, the switch SW7for a menu or confirmation (OK), the switch SW8for the LCD monitor20, the switch SW9for upward movement, the switch SW10for rightward movement, the switch SW11for downward movement, the switch SW12for leftward movement, and the switch SW13for power, are provided at a back surface of the imaging device1. The switch SW9may also have a function to set the flash lamp13. The switch SW11may also have a function to set macro. The SW12may also have a function to display an image on the LCD monitor20for confirmation.

Reference toFIG. 2C, the processor34includes a CPU and components of the imaging device1are controlled by the processor34. The SDRAM33, the RAM35, the built-in memory36, and the ROM37are provided outside the processor34and connected to the processor34via the BUS-line (not shown in the drawings). The built-in memory36stores an obtained image. The ROM37includes a control program, parameters or the like for actualizing functions of the imaging device1of the embodiment.

The SDRAM33stores a RAW-RGB image (to which white-balance correction and γ correction are performed), a YUV image (conversed to luminance data and color-difference data), a JPEG image (to which JPEG compression is performed) or the like in addition to the obtained original image.

In this embodiment, when the switch SW13for power is ON, the control program stored in the ROM37is loaded into memory in the processor34or a memory connected to the processor34to be executed. The components of the imaging device1are controlled by the control program. In this embodiment, when the control program is executed, the RAM35may be used for an operational memory for the control program. It means that the control program or parameters are appropriately written in the RAM35and read from the RAM35. The following processes are mainly performed by the processor34by executing the control program.

The motor driver175of the lens-barrel unit17is controlled by the CPU block344of the processor34. The motor driver175drives the zoom motor171b, the focus motor172b, the aperture motor173b, and the mechanical shutter motor174bto drive the zoom lens171a, the focus lens172a, the aperture173a, and the mechanical shutter174a, respectively.

A user operates the switch SW3for wide-angle zoom or the switch SW4for telephoto zoom to have the zooming optical system171and the focusing optical system172focus an image of an object on light accepting surfaces of the CCD31. The focused image of the object is converted to an image signal by the CCD31to be output to the F/E-IC32.

In the F/E-IC32, the CDS circuit321performs a correlated double sampling on the image signal. Then, the AGC circuit322performs an automatic gain control on the image signal output from the CDS circuit321. The A/D converter323converts the image signal, which is an analog signal output from the AGC circuit322, to a digital image signal. It means that the F/E-IC32inputs the analog image signal output from the CCD31, performs a predetermined operation such as reducing noise, an automatic gain control or the like, converts the analog image signal to the digital image signal and then outputs the digital image signal to the CCD1signal processing block342of the processor34.

The TG324generates the driving timing signal for sampling of the image signal by the F/E-IC32based on the vertical driving pulse VD and the horizontal driving pulse HD which are fed back from the CCD1signal processing block342of the processor34.

The I2C block341performs a high speed communication between the ICs by a serial communication. The CCD1signal processing block342performs a white-balance correction or a γ correction on the digital image signal output from the CCD31via the F/E-IC32. Further, the CCD1signal processing block342supplies the vertical driving pulse VD and the horizontal driving pulse HD to the TG324, as described above. The CCD2signal processing343converts the input image signal to luminance data, the color-difference data or the like by a filtering process. The CPU block344controls the operations of components of the imaging device1. The functions of the CPU block344according to the embodiment will be explained later in detail.

The local SRAM345temporarily stores various data necessary to control components of the imaging device1. The USB block346performs a USB communication between an external device such as a personal computer or the like. The serial block347performs a serial communication between an external device such as a personal computer or the like. The JPEG codec block348performs a JPEG compression or a JPEG expansion.

The resize block349reduces or enlarges an image to a predetermined size, performs interpolation or the like. The TV signal display block350converts the image signal to a video signal to be displayed on an external display device such as a liquid crystal monitor, TV or the like. The memory card controller block351controls the memory card53on which the obtained image is stored.

The CPU block344of the processor34is connected to the F/E-IC32, the motor driver175, the voice recording circuit381, the audio play circuit391, the flash lamp circuit40that controls the flash lamp13, the ranging unit15, and the sub CPU42to control these components.

For the voice input unit38, a voice signal obtained by the microphone383is amplified by the microphone amplifier382, converted to a digital signal by the voice recording circuit381, and stored in the built-in memory36, the memory card53or the like, based on control by the CPU block344. For the audio play unit39, voice data previously stored in the ROM37or the like is converted by the audio play circuit391based on a control by the CPU block344, amplified by the audio amplifier392, and output from the speaker393.

The ranging unit15includes a two-dimensional sensor or the like as a ranging sensor and measures a distance to a predetermined object within a field of view of the imaging device1by the two-dimensional sensor or the like. The ranging unit15may measure the distance using plural two-dimensional sensors or the like. For example, the ranging unit15may measure the distance using parallax between images obtained by the optical lenses such as the zoom lens171aand the focus lens172aand image sensors of the CCD31or the like, by triangulation or the like.

The sub CPU42is connected to the sub LCD11via the LCD driver41, the LED for AF18, the LED for flash19, the remote control light receiver16, the operational key unit43, and the buzzer44to control these components. The sub CPU42monitors signal inputs to the remote control light receiver16or the operational key unit43. The operational key unit43may be composed of the switches SW1to SW13.

The acceleration sensor45, mounted on a Printed Circuit Board (PCB), is connected to the I2C block341. The acceleration sensor45outputs acceleration data of two orthogonal axes in an X-direction and in a Y-direction and temperature data. The acceleration sensor45calculates an inclination of the imaging device1such as a roll angle or a pitch angle based on the acceleration data to output to the I2C block341and to be displayed on the LCD monitor20or the like. The roll angle θ with respect to a horizontal plane can be obtained by the following equation (1).
θ[deg]=180/π×arctan((Y0−G0)/(X0−G0))  (1)
Here, X0expresses acceleration data when the gravity is zero.

In this embodiment, it is assumed that the roll angle θ of the imaging device1is 0° when the imaging device1takes the position as shown inFIG. 1BorFIG. 1C. Further, it is assumed that the imaging device1is inclined in a positive direction when the LCD monitor20is inclined in a clockwise direction, while it is assumed that the imaging device1is inclined in a negative direction when the LCD monitor20is inclined in a counterclockwise direction.

The USB block346is connected to the USB connector46. The serial block347is connected to the RS-232C connector48via the serial driver circuit47. The imaging device1communicates with an external device such as a personal computer or the like via the USB block346or the serial block347.

The TV signal display block350is connected to the LCD driver49and the video amplifier50. The LCD driver49drives the LCD monitor20. The video amplifier50amplifies the video signal and performs an impedance matching. The LCD monitor20is connected to the LCD driver49. The video jack51to be connected with an external monitor device such as a television or the like is connected to the video amplifier50.

The LCD monitor20is provided to display an object while obtaining an image, an obtained image, an image stored in the memory card53, the built-in memory36or the like. The LCD monitor20may be composed to have a function of an input device such as a touch panel or the like. In such a case, a predetermined object may be specified by touching the image displayed on the LCD monitor20, or various instructions can be input through the LCD monitor20.

The memory card slot52is connected to the memory card controller block351. The imaging device1may store and read out an image in the memory card53which is inserted in the memory card slot52. The lens-barrel unit17, the CCD31, the F/E-IC32, and the CCD1signal processing block342of the imaging device1function as an image obtaining unit.

Instead of the CCD31, Complementary Metal Oxide Semiconductors (CMOS) may be used. In such a case, a CMOS1signal processing block and a CMOS2signal processing block may be used instead of the CCD1signal processing block342and the CCD2signal processing block343, respectively.

(Operation of Electronic Image Stabilization)

Before describing an operation of electronic image stabilization by the imaging device1of the embodiment, a general operation of electronic image stabilization will be explained for explanation purposes.

FIG. 4is a flowchart showing an example of an operation of electronic image stabilization.

First, plural continuous images, in this case four images, for example, of an object are obtained (S01). Then, the four images obtained in step S01are reduced to four reduced images, each having a predetermined size, for detecting a motion vector respectively (S02).

Thereafter, a motion vector is detected based on the four reduced images obtained in step S02by aligning the reduced images in a time-series and comparing corresponding points of the object in respective successive images (S03). Then, the images obtained in step S01, each having the original size, are synthesized based on the motion vector obtained in step S03to obtain a synthesized image (S04). The synthesized image is then stored in the memory or the like (S05).

With this method, when performing the electronic image stabilization, four continuous images are obtained. Then, four reduced images having a size smaller than the original obtained images are generated. Thereafter, a relative motion vector is detected based on the reduced images. Then, the original obtained images are synthesized based on the detected motion vector to generate the synthesized image. Thereafter, the synthesized image is stored in a memory unit such as the built-in memory36, the memory card53inserted in the memory card slot52, or the like.

FIG. 5is a view for explaining the electronic image stabilization. Four images611to614which are obtained successively are respectively reduced to a predetermined size to be reduced images621to624, respectively. The reduced images621to624are overlapped (63) and used for detecting the motion vector for the electronic image stabilization.

The original images611to614are synthesized to correct the blur or the like caused by hand movement or camera shake when obtaining the images611to614based on the motion vector obtained based on the reduced images621to624. The synthesized image64may be stored in a memory unit such as a built-in memory, a memory card or the like.

FIG. 6is a view for explaining synthesizing of images.

In this example as shown inFIG. 6, viewing angles of the four images611to614do not match with each other. However, when synthesizing the images611to614, one of the images, in this case the image611, which is obtained first, is set as a reference. The area where the two or more images overlap is synthesized. In other words, the area where the two or more images overlap becomes a synthesizable area, so that it is not necessary to use the four images for all the areas. Therefore, the one to four images are partially used to obtain the synthesized image.

InFIG. 6, the synthesized image71obtained by the images611to614is also shown as expressed by dots.

Generally, when detecting the motion vector, the larger the image size is, the higher the accuracy of the motion vector becomes so that the blur or the like can be precisely corrected. However, when detecting the motion vector by images of a larger size, it takes a longer time. Therefore, generally, the size of images for detecting the motion vector is set equal as a Video Graphics Array (VGA: for example, 640 (horizontal)×480 (vertical) and total pixel number of 307200).

However, there may be a case where a motion vector with a higher accuracy is required, or a case where a faster speed is required based on a condition in obtaining an image such as a zoom position or the like of the imaging device1, or a obtained image such as a kind of an object included in the image or the like. Therefore, it is desirable to arbitrary change the image size of the images (hereinafter referred to as “reduced images”) used for detecting the motion vector based on the requirement.

In this embodiment, the CPU block344determines the image size of reduced images used for detecting a motion vector based on photographing information such as setting information of the imaging device1in obtaining an image, or characteristic information of an obtained image.

The photographing information may be capable of defining or indicating a necessary accuracy for the motion vector. The CPU block344determines the image size of reduced images for detecting the motion vector in accordance with the necessary accuracy of the photographing information such that the higher the necessary accuracy, the larger the image size becomes.

Specifically, when performing electronic image stabilization, when the CPU block344detects that the switch for release SW1is pushed, four continuous (successive) images are obtained and stored in the SDRAM33.

Then, four reduced images having the image size determined based on the photographing information are generated by reducing the obtained four images, respectively. Thereafter, a motion vector is detected based on the reduced images. Subsequently, a synthesized image is generated based on the obtained images and the detected motion vector.

The generated synthesized image is stored in the built-in memory36or the memory card53inserted in the memory card slot52.

(Functional Structure of CPU Block344)

The structure of the CPU block344of the embodiment will be explained in detail.FIG. 3is a functional block diagram of the CPU block344.

The CPU block344includes an image obtaining control unit344a, a size determining unit344b, a motion vector detection unit344c, an image synthesis unit344d, and an image storage control unit344e.

The image obtaining control unit344acontrols the components of the imaging device1to obtain continuous plural images of a predetermined number, which is previously set, from the CCD31.

The image obtaining control unit344amay determine whether an electronic image stabilization mode is selected or not before obtaining the plural images. Then, if the electronic image stabilization mode is selected, the image obtaining control unit344acontrols the components of the imaging device1to obtain the plural images. On the other hand, if the electronic image stabilization mode is not selected, the image obtaining control unit344amay control the components of the imaging device1to obtain a single image, as a normal mode. The user of the imaging device1may set the mode by operating the switch SW2for mode dial.

The size determining unit344bdetermines the image size of reduced images for detecting a motion vector based on the photographing information such as the setting information of the imaging device1in obtaining the continuous images, or the characteristic information of the obtained images.

Specifically, the setting information of the imaging device1in obtaining an image may be a zoom position, ISO speed, a focus position, F-number of lens, parameter for white-balance, γ curve used in γ correction, parameter for aperture compensation, parameter for noise reduction, parameter for distortion correction or the like.

The setting information of the imaging device1in obtaining the images may be obtained and stored with the obtained images.

For the setting information, the zoom position may be used. When the zoom position is at a telephoto side, large blurring may easily occur and an object included in an obtained image may be larger so that precise detection of a motion vector is necessary. However, on the other hand, when the zoom position is at a wide-angle side, large blurring does not easily occur and an object included in an obtained image may be small, so that a precise detection of a motion vector is not as necessary. Therefore, it is desirable to set the image size, reduction ratio or the like, of reduced images based on the zoom position.

The characteristic information of the images may be, for example, the contrast of an obtained image, or whether a predetermined shape or object, such as a face of a person, a specific structure, an animal or the like, is included in an obtained image.

For the characteristic information, the contrast of an obtained image may be used. When the contrast of the obtained image is high, the boundary between objects included in the image becomes clearer so that a precise detection of a motion vector is necessary. Therefore, it is desirable to set the image size, the reduction ratio or the like, of reduced images based on the contrast of the obtained image.

Further, the size determining unit344bmay determine the image size of reduced images using both the setting information and the characteristic information.

Although not shown in the drawings, the imaging device1may store a table in which photographing information (setting information or characteristic information) and image sizes of reduced images (a reduction ratio, an enlargement ratio, or the like) for detecting a motion vector corresponding with each other. The image size may be expressed as resolution as well.

The size determining unit344bgenerates reduced images, having the determined image size, based on the obtained images. At this time, the size determining unit344bmay control the resize block349to generate the reduced images by reducing the obtained images to the determined image size.

The motion vector detection unit344cdetects a motion vector of the reduced images. With this, an appropriate motion vector can be detected based on the setting information of the imaging device1or the characteristic information of the obtained images.

The image synthesis unit344dsynthesizes the obtained images based on the motion vector detected by the motion vector detection unit344cto output a single synthesized image or plural synthesized images.

The image storage control unit344econtrols storage of the images obtained by the image obtaining control unit344a, the reduced images generated by the size determining unit344b, the synthesized image generated by the image synthesis unit344dor the like in the SDRAM33, the RAM35, the built-in memory36, the memory card53inserted in the memory card slot52, or the like.

The components of the CPU block344actualize the electronic image stabilization by the imaging device1of the embodiment as will be explained later.

The method of detecting the motion vector or synthesizing the images may not be limited to the above described method, and may be performed in accordance with techniques disclosed in Patent Document 1, U.S. Pat. No. 7,773,819, Japanese Laid-open Patent Publication No. 2006-157568, and U.S. Pat. No. 7,903,155 or the like.

The example of the operation of the electronic image stabilization by the imaging device1of the embodiment is explained in detail.

First Example

FIG. 7is a flowchart showing a first example of an operation of electronic image stabilization by the imaging device1of the embodiment.

Generally, when obtaining an image, when the image is obtained with telephoto, compared with a case when the image is obtained with wide-angle, an object becomes larger in the image and the image becomes a more detailed graphic. Therefore, in this case, when the image is obtained with telephoto, it is necessary to detect the motion vector with a higher accuracy and it may be better to set the image size of the reduced images for obtaining the motion vector larger compared with the case when the image is obtained with wide-angle.

Thus, in the first example, the image size of the reduced images for obtaining a motion vector is set based on the zoom position of the imaging device1where the image size of the reduced images is set larger when the zoom position is set at the telephoto side compared with the case when the zoom position is set at the wide-angle side. The zoom position may be set in the setting information so that the size determining unit344bof the CPU block344can determine the zoom position of the imaging device1based on the setting information.

Reference toFIG. 7, when the switch for release SW1is pushed, the image obtaining control unit344aof the CPU block344determines whether the electronic image stabilization mode is selected for the imaging device1(S11). When it is determined that the electronic image stabilization mode is selected (YES in S11), the image obtaining control unit344acontrols the components of the imaging device1to obtain plural images, four images in this example, (S12), and stores the obtained images (image data in an image signal form) in the SDRAM33, for example. At this time, the setting information including the zoom position is also obtained and stored in the SDRAM33with the obtained images in correspondence with each other.

The size determining unit344bof the CPU block344determines the image size of reduced images for detecting a motion vector, which will be generated by reducing the obtained images stored in the SDRAM33.

In this example, the size determining unit344bdetermines the image size based on the zoom position of the imaging device1when the four images are obtained. Specifically, the size determining unit344brefers to the setting information stored in the SDRAM33to obtain the zoom position. Further, the imaging device may include a table in which zoom positions and image sizes of reduced images for detecting a motion vector correspond with each other. By referring to the table, the size determining unit344bdetermines the image size of images for motion vector detection (S13).

Then, the size determining unit344bof the CPU block344reduces the size of the four images obtained in step S12to have the image size determined in step S13to generate reduced images (S14). Subsequently, the motion vector detection unit344cdetects a motion vector based on the images for motion vector detection (S15).

Then, the image synthesis unit344dof the CPU block344synthesizes the images obtained in step S12based on the motion vector obtained in step S15to generate a synthesized image (S16).

When it is determined that the electronic image stabilization mode is not selected (NO in S11), when the switch for release SW1is pushed, the image obtaining control unit344aof the CPU block344obtains a single image in a normal mode (S18).

The synthesized image generated in step S16, or the image obtained in step S18is stored in the built-in memory36, the memory card53inserted in the memory card slot52or the like (S17).

As described above, according to the first example of the embodiment, the CPU block344is capable of generating an image appropriately corrected by the electronic image stabilization with an appropriate accuracy with a higher speed based on the zoom position (setting information) of the imaging device1.

The number of images for the continuous images used for the operation of the electronic image stabilization is not limited to four, as long as it is two or more.

Second Example

FIG. 8is a flowchart showing a second example of the operation of the electronic image stabilization by the imaging device1of the embodiment.

Generally, when obtaining an image, when the image is obtained with a higher ISO speed, compared with a case when the ISO speed is lower, the signal-noise ratio (S/N ratio) of image noise tends to become worse (lower) or the shutter speed tends to become faster. Further when the S/N ratio of the image noise becomes lower, in other words when the image noise becomes larger, the blur in the image tends to become indistinctive because of the influence by the noise.

However, when the image is obtained with a lower ISO speed, the S/N ratio of image noise tends to become higher so that it is necessary to detect the motion vector with a higher accuracy and it may be better to set the image size of the reduced images for obtaining the motion vector larger.

Thus, in the second example, the image size of reduced images for obtaining a motion vector is set based on the ISO speed of obtained images where the lower the ISO speed becomes the larger the image size becomes. The ISO speed may be set in the setting information so that the size determining unit344bof the CPU block344can determine the ISO speed of the imaging device1based on the setting information.

Reference toFIG. 8, when the switch for release SW1is pushed, the image obtaining control unit344aof the CPU block344determines whether the electronic image stabilization mode is selected for the imaging device1(S21). When it is determined that the electronic image stabilization mode is selected (YES in S21), the image obtaining control unit344acontrols components of the imaging device1to obtain plural images, four images in this example, when the switch for release SW1is pushed (S22), and stores the obtained images in the SDRAM33, for example. At this time, setting information including the ISO speed is also obtained and stored in the SDRAM33with the obtained images in correspondence with each other.

The size determining unit344bof the CPU block344determines the image size of reduced images for detecting a motion vector, which will be generated by reducing the obtained images stored in the SDRAM33.

In this example, the size determining unit344bdetermines the image size based on the ISO speed of the imaging device1when the four images are obtained. Specifically, the size determining unit344brefers to the setting information stored in the SDRAM33to recognize the ISO speed. Further, the imaging device may include a table in which ISO speeds and image sizes of reduced images for detecting a motion vector correspond with each other. By referring to the table, the size determining unit344bdetermines the image size of images for motion vector detection (S23).

Then, the size determining unit344bof the CPU block344reduces the size of the four images obtained in step S22to have the image size determined in step S23to generate images for motion vector detection (S24). Subsequently, the motion vector detection unit344cdetects a motion vector based on the images for motion vector detection (S25). Then, the image synthesis unit344dof the CPU block344synthesizes the images obtained in step S22based on the motion vector obtained in step S25to generate a synthesized image (S26).

When it is determined that the electronic image stabilization mode is not selected (NO in S21), when the switch for release SW1is pushed, the image obtaining control unit344aof the CPU block344obtains a single image in a normal mode (S28).

The synthesized image generated in step S26, or the image obtained in step S28is stored in the built-in memory36, the memory card53inserted in the memory card slot52or the like (S27).

As described above, according to the second example of the embodiment, the CPU block344is capable of generating an image appropriately corrected by the electronic image stabilization with an appropriate accuracy with a higher speed based on the ISO speed (setting information) of the imaging device1.

The number of images for the continuous images used for the operation of the electronic image stabilization is not limited to four, as long as it is two or more.

The setting information is not limited to the zoom position or the ISO speed, and other setting information such as conditions of the imaging device1when taking an image as described above may be used as well. Further, a combination of plural kinds of setting information may also be used.

Third Example

FIG. 9is a flowchart showing a third example of the operation of the electronic image stabilization by the imaging device1of the embodiment.

Generally, when the contrast of the obtained image is high, the boundary between a bright part and a dark part becomes clearer so that it is easily recognized if there is a positional displacement when synthesizing the obtained images. Therefore, in this case, it is necessary to detect the motion vector with a higher accuracy and it may be better to set the image size of the reduced images for obtaining the motion vector larger.

When, on the other hand, the contrast of the obtained image is lower, the boundary between a bright part and a dark part becomes unclear so that it is hardly recognized even if there is just a slight positional displacement when synthesizing the obtained images. Therefore, in this case, it is not necessary to detect the motion vector with a higher accuracy and it may be better to set the image size of the reduced images for obtaining the motion vector smaller to shorten the operation period.

Thus, in the third example, the image size of reduced images for obtaining a motion vector is set based on the contrast of obtained images where the higher the contrast, the larger the image size becomes.

Reference toFIG. 9, when the switch for release SW1is pushed, the image obtaining control unit344aof the CPU block344determines whether the electronic image stabilization mode is selected for the imaging device1(S31). When it is determined that the electronic image stabilization mode is selected (YES in S31), the image obtaining control unit344acontrols the components of the imaging device1to obtain plural images, four images in this example, (S32), and stores the obtained images in the SDRAM33, for example.

The size determining unit344bof the CPU analyzes one of the four images stored in the SDRAM33, which may be the image obtained first, to determine the contrast. Then, the size determining unit344bdetermines the image size of reduced images for detecting a motion vector based on the contrast (S33).

Alternatively, the contrast of the image may be determined based on plural images among the four images, for example, the images obtained first and last (the first image and the fourth image). In this case, the contrast of each of the images is determined first, then, the average is calculated based on the contrasts of the images. With this, the contrast of the plural images can be obtained with a higher accuracy.

Then, the size determining unit344bof the CPU block344reduces the size of the four images obtained in step S32to have the image size determined in step S33to generate reduced images (S34). Subsequently, the motion vector detection unit344cdetects a motion vector based on the images for motion vector detection (S35).

Then, the image synthesis unit344dof the CPU block344synthesizes the images obtained in step S32based on the motion vector obtained in step S35to generate a synthesized image (S36).

When it is determined that the electronic image stabilization mode is not selected (NO in S31), when the switch for release SW1is pushed, the image obtaining control unit344aof the CPU block344obtains a single image in a normal mode (S38).

The synthesized image generated in step S36, or the image obtained in step S38is stored in the built-in memory36, the memory card53inserted in the memory card slot52or the like (S37).

As described above, according to the third example of the embodiment, the CPU block344is capable of generating an image appropriately corrected by the electronic image stabilization with an appropriate accuracy with a higher speed based on the contrast (characteristic information) of the image.

The number of images for the continuous images used for the operation of the electronic image stabilization is not limited to four, as long as it is two or more.

Fourth Example

FIG. 10is a flowchart showing a fourth example of the operation of the electronic image stabilization by the imaging device1of the embodiment.

Generally, when a face of a person or the like is included in the obtained image, it may be necessary to maintain the image precisely. Therefore, in this case, it is necessary to detect the motion vector with a higher accuracy and it may be better to set the image size of the reduced images for obtaining the motion vector larger.

Reference toFIG. 10, when the switch for release SW1is pushed, the image obtaining control unit344aof the CPU block344determines whether the electronic image stabilization mode is selected for the imaging device (S41). When it is determined that the electronic image stabilization mode is selected (YES in S41), the image obtaining control unit344acontrols the components of the imaging device1to obtain plural images, four images in this example, (S42), and stores the obtained image in the SDRAM33, for example.

The size determining unit344bof the CPU analyzes one of the four images stored in the SDRAM33, which may be the image obtained first, to determine whether a face of a person is included. Then, the size determining unit344bdetermines the image size of reduced images for detecting a motion vector based on whether the face of a person is included (S43). Specifically, when it is determined that the face of a person is included, the size determining unit344bsets the image size of reduced images larger.

In step S43, when it is determined that the face of a person is included, the size determining unit344bmay determine the image size of reduced images based on the area size for the face, the position of the face or the number of faces.

Alternatively, whether a face of a person is included may be determined based on all of the obtained images and when it is determined that the face of a person is included in at least one of the images, the size determining unit344bmay set the image size of reduced images larger. Further alternatively, when it is determined that a face of a person is included in the plural images, the size determining unit344bmay set the image size of reduced images based on one of the plural images, for example, an earlier one.

The characteristic information may not be limited to a face of a person, but a predetermined object other than a face of a person may be previously set, and the size determining unit344bmay determine the image size of reduced images based on whether the predetermined object is included in the image or not.

Then, the size determining unit344bof the CPU block344reduces the size of the four images obtained in step S42to have the image size determined in step S43to generate reduced images (S44). Subsequently, the motion vector detection unit344cdetects a motion vector based on the images for motion vector detection (S45).

Then, the image synthesis unit344dof the CPU block344synthesizes the images obtained in step S42based on the motion vector obtained in step S45to generate a synthesized image (S46).

When it is determined that the electronic image stabilization mode is not selected (NO in S41), when the switch for release SW1is pushed, the image obtaining control unit344aof the CPU block344obtains a single image in a normal mode (S48).

The synthesized image generated in step S46, or the image obtained in step S48is stored in the built-in memory36, the memory card53inserted in the memory card slot52, or the like (S47).

As described above, according to the fourth example of the embodiment, the CPU block344is capable of generating an image appropriately corrected by the electronic image stabilization with an appropriate accuracy with a higher speed based on whether the face of a person is included in the obtained image (characteristic information).

The number of images for the continuous images used for the operation of the electronic image stabilization is not limited to four, as long as it is two or more.

The characteristic information may not be limited to the contrast of the image or whether the face of a person is included in the image and other characteristic information of the image may be used as well. Further, a combination of plural kinds of characteristic information may also be used.

Further, the information as described in the first to fourth examples may be arbitrary combined to obtain a same effect. Therefore, not limited to the above examples, a combination of any setting information and any characteristic information may be used for the operation of the electronic image stabilization of the embodiment.

Further, for example, when the zoom position of the imaging device1is at the telephoto side where a precise detection of a motion vector is necessary, the reduction ratio of the image size of reduced images may be ½ size for both the vertical direction and the horizontal direction (reduction ratio of 25%). When, on the other hand, the zoom position is at the wide-angle side where a precise detection of a motion vector is not as necessary, the reduction ratio may be ¼ for both the vertical direction and the horizontal direction (reduction ratio of 6.25%). These are just examples and the reduction ratio or the enlargement ratio may be arbitrary set based on various photographing information.

Further alternatively, the number of the images used for detecting the motion vector, or generating the synthesized image may be arbitrary set provided the number is two or more. Further, alternatively, the synthesizable area may be based on two or more images. Further, alternatively, the reference image when used at synthesizing the images may be the image other than the image which is obtained first.

The operation of the electronic image stabilization using the imaging device1as described above may be actualized by a program executable by a computer. By installing the program in a device including the hardware structure as shown inFIG. 2AtoFIG. 2Cand executing the program by the processor34or the like, the operation of the imaging device1can be actualized.

Further, the table in which photographic information (setting information or characteristic information) and the image sizes of reduced images (a reduction ratio, an enlargement ratio, or the like) for detecting a motion vector corresponding with each other may be included not in the imaging device1, but instead may be stored in an external apparatus with which the imaging device1is capable of communication. In this case, the imaging device1may obtain the image size from the external apparatus by sending the photographic information to the external apparatus.

As described above, according to the embodiment, an image appropriately corrected by the electronic image stabilization can be obtained. Specifically, the size of reduced images for detecting the motion vector is determined based on the photographic information which is capable of defining a necessary accuracy for a motion vector so that the image size of the reduced images for detecting a motion vector is determined in accordance with the necessary accuracy of the photographing information such that the higher the necessary accuracy, the larger the image size becomes. With this structure, the motion vector is appropriately detected in regard to accuracy and speed. With this, an appropriate operation of electronic image stabilization can be actualized.

According to the embodiment, an image appropriately corrected by electronic image stabilization can be obtained.

The present application is based on Japanese Priority Application No. 2011-061631 filed on Mar. 18, 2011, the entire contents of which are hereby incorporated herein by reference.