Image capturing apparatus having a control unit controlling switching unit such that first image is displayed in case a predetermined motion is detected and a composite image is displayed in case motion is not detected, control method thereof, and storage medium

An image capturing apparatus, comprises an image capturing unit configured to capture an image of an object; a switching unit configured to switch a display unit so that either a first image captured in a first exposure time or a composite image composed of a second image captured in a second exposure time shorter than the first exposure time is displayed; a detection unit configured to detect motion of the apparatus; and a control unit configured to control the switching unit so that the first image is displayed in a case where a predetermined motion has been detected by the detection unit and the composite image is displayed in a case where the predetermined motion is not detected.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a control technique of an image capturing apparatus that switches so that a live image or a composite image composed of images, each of which has a short exposure time, is displayed.

2. Description of the Related Art

In recent years, image capturing apparatuses such as digital cameras have been proposed that generate a single composite image composed of multiple images and provide this to the user. A first object of generating a composite image is for expanding the dynamic range. In other words, this is because by varying the exposure while capturing multiple images and composing images having different exposures, it is possible to generate an image having little highlight-detail loss (overexposure) and shadow-detail loss (underexposure) and a wide dynamic range. A second object is greater high-sensitivity. This is because by composing multiple images, the (random) noise component contained in the image is suppressed and the SN ratio of the composite image can be improved.

A point of caution when generating the above-mentioned composite image is that, depending on the object and capture conditions, the composite image can engender a visual sense of strangeness (unnaturalness). Furthermore, in composing the images it is necessary to compose after correcting positional shifting produced between the multiple images, but sometimes this positional shifting cannot be corrected sufficiently depending on the object or the capture conditions such that a blurred image is generated. In this way, sometimes the composite image is undesirably different from the image intended by the user.

For example, Japanese Patent Laid-Open No. 2009-088927 has optimal internal settings for a normal shooting mode (not performing image composing) and a composite image mode, and provides optimal image quality with these modes.

However, optimal settings do not necessarily match the user's preferences and in this case the user will be required to experiment twice with the normal shooting mode and with composite image shooting. Or embodiments are conceivable in which, depending on the product, shooting is performed automatically with the normal shooting mode and image composing with a one-time shutter release, thereby allowing the user to select afterward.

Either way, since the composite image cannot be confirmed until after shooting (after shutter release), it will be desirable for the user to be able to easily confirm the composite image on the display unit prior to actual shooting.

However, problems occur also in cases where the composite image is always displayed prior to shooting. For example, as shown inFIG. 3, with composite images during panning or zooming, depending on the conditions, sometimes a portion of the composite image will become dark or blurred. Accordingly, since zooming and panning often occurs while the user adjusts the composition or angle of view prior to shooting, displaying a single image without displaying a composite image is better for the tracking characteristics of the displayed image for motion and is preferable for the purpose of adjusting the composition and the angle of view.

On the other hand, after the user has finished panning or zooming and immediately prior to shooting, it is preferable to display the composite image so as to enable a finished composite image to be confirmed prior to shooting.

SUMMARY OF THE INVENTION

The present invention has been made in consideration of the aforementioned problems, and realizes a control technology capable of avoiding risks of shooting failures by performing switching so that a single image or a composite image is displayed in response to motion or operations of the camera prior to shooting.

In order to solve the aforementioned problems, the present invention provides an image capturing apparatus, comprising: an image capturing unit configured to capture an image of an object; a switching unit configured to switch a display unit so that either a first image captured in a first exposure time or a composite image composed of a second image captured in a second exposure time shorter than the first exposure time is displayed; a detection unit configured to detect motion of the apparatus; and a control unit configured to control the switching unit so that the first image is displayed in a case where a predetermined motion has been detected by the detection unit and the composite image is displayed in a case where the predetermined motion is not detected.

In order to solve the aforementioned problems, the present invention provides a control method of an image capturing apparatus which has an image capturing unit configured to capture an image of an object; and a detection unit configured to detect motion of the apparatus, the method comprising: a switching step of switching a display unit so that either a first image captured in a first exposure time or a composite image composed of a second image captured in a second exposure time shorter than the first exposure time is displayed; and a control step of controlling the switching of the display in the switching step so that the first image is displayed in a case where a predetermined motion has been detected by the detection unit and the composite image is displayed in a case where the predetermined motion is not detected.

According to the present invention, risks of shooting failures can be avoided by performing switching so that a single image or a composite image is displayed in response to motion or operations of the camera prior to shooting.

DESCRIPTION OF THE EMBODIMENTS

Embodiments of the present invention will be described in detail below. The following embodiments are merely examples for practicing the present invention. The embodiments should be properly modified or changed depending on various conditions and the structure of an apparatus to which the present invention is applied. The present invention should not be limited to the following embodiments. Also, parts of the embodiments to be described later may be properly combined.

Configuration of Image Capturing Apparatus

Description is given with reference toFIG. 1regarding a configuration of an image capturing apparatus according to an embodiment of the present invention. It should be noted that in the following description an image that is generated by composing multiple images is referred to as a composite image and an image that has been captured is referred to as a single image.

For example, the image capturing apparatus according to the present embodiment is a digital camera having a function of composing one image from multiple images. And in the present embodiment, a display unit is switched in response to a state change of the apparatus so as to display either a first image (single image) captured in a first exposure time (frame) or a composite image composed of a second image captured in a second exposure time shorter than the first exposure time.

InFIG. 1, the image capturing apparatus realizing the present invention mainly includes a control unit101, an image capturing unit102, an image reducing unit103, an image composing unit104, a switching device105, a display unit106, and an operation unit107, and a motion detecting device108.

The control unit101is a microcomputer for example, and outputs a control signal to each of the above-mentioned units102to108as well as receiving a signal from each of the units102to108indicating an operational status and the like.

The operation unit107includes a user interface relating to control of the image capturing apparatus represented by operation keys, a touch panel, a zoom button and a shutter button for example. User operations in regard to the image capturing apparatus are inputted to the control unit101through the operation unit107, and actions of each of the units102to108are determined by receiving a control signal from the control unit101.

The image capturing unit102is a CMOS image sensor for example, and captures images at 60 frames per second for example during normal shooting. Hereinafter, although description is given assuming a frame rate of 1/60 seconds during shooting, the present invention is not limited to this frame rate.

Normally, prior to actual shooting (an operation of actually releasing the shutter), the user views an image on the display unit106and carries out settings such as angle of view and exposure. This period is referred to as a pre-shot period.

Based on an electronic shutter function, the image capturing unit102can freely vary the exposure time within the 1/60 seconds. Furthermore, the exposure time is variable in frame units. It should be noted that electronic shutter signifies a function that achieves an action equivalent to a shutter release by electronically changing the charge accumulation period after all the charges of the photodiodes of the CMOS image sensor have been cleared out once.

In the pre-shot period, driving of the image capturing unit102is carried out as inFIG. 2for example.

Given that a maximum 1/60 second charge accumulation is possible in a single frame, in a case viewed using a two frame unit, charge accumulation is carried out for a specific time for one of the frames (an A frame) by driving the image capturing unit102as an electronic shutter where the maximum 1/60 second charge accumulation is possible. For the other frame (a B frame), the charge accumulation (exposure amount) is different between frames using the electronic shutter. The amount of charge accumulation for the B frame is determined in response to purposes such as dynamic range expansion and SN ratio improvement in the post-processing of image composing processing.

The A frame is used for finally generating a single image. The B frame is used for finally generating a composite image.

The data size of both the A frame and the B frame is reduced by the image reducing unit103. This is because the image data obtained from the image capturing unit102is extremely large compared to the pixel size of the display unit106.

After undergoing reduction processing by the image reducing unit103, the B frame is outputted to the image composing unit104. The image composing unit104composes image data of each frame of the B frame.

The image composing unit104is a signal processing circuit having a function that is capable of composing image data of a large number of pixels obtained from the image capturing unit102at the time of actual shutter release. It should be noted that since the data amount of the reduced image data from the image reducing unit103is small, the processing load required for image composing is not large and also the time required for image composing processing is short. The image data composed of B frame image data by the image composing unit104is referred to as a B′ frame.

The switching device105inputs the A frame from the image reducing unit103and the B′ frame from the image composing unit104and switches the image data of the frame to be outputted to the display unit106. The control unit101determines whether image data of the A frame or the B′ frame should be outputted.

For example, the motion detecting device108is a sensor that is provided within the image capturing apparatus for detecting angular velocity in three axial directions (X, Y, and Z), and is capable of detecting hand wobbling and panning actions of the shooter. Furthermore, the motion detecting device108carries out detection of motion vectors of an object from the reduced image data generated by the image reducing unit103, and calculates the movement velocity and movement direction of the object and outputs the calculation results to the control unit101.

The control unit101controls the switching device105in a following manner.

For example, in a case where the shooter is carrying out a pan or a zoom, the state of the pan can be obtained from the motion detecting device108and information of the zoom can be obtained from the operation unit107. Here, the switching device105outputs A frame image data. This is because when composite image data is outputted during panning or zooming, there is a possibility that a portion of the image will become dark or that blur will be generated in the image as shown inFIG. 3.

Furthermore, the switching device105outputs A frame image data also in a case where the hand wobbling detected by the motion detecting device108is large. This is because when hand wobbling is large there is a possibility that image positional alignment will not be performed well between frames when composing the images and blur will be generated in the composite image.

Furthermore, the switching device105outputs A frame image data also in a case where the movement amount of the object detected by the motion detecting device108is large. This is because when there is large motion of the object, there is a possibility of a case of a portion of the image becoming dark or that blur will be generated in the image in a same manner as described earlier.

When the above-mentioned zooming, panning, hand wobbling, and object movement amount are within a certain range, the switching device105outputs B′ frame image data.

By switching the display of images for each frame as described above, the A frame image data is displayed on the display unit106when the motion of the object is fast for example when the shooter is zooming or panning. While viewing that image, the user can easily determine the composition and angle of view or the like by carrying out operations relating to shooting.

On the other hand, for situations other than those described above, that is, in situations where the shooting conditions are in order such as a case where zooming and panning have been stopped and the shooter's desired composition and angle of view have been decided or a case where movement of the object has stopped, the B′ frame image data is displayed on the display unit106. Due to this, the user can confirm the composite image before shooting and can avoid situations such as viewing the composite image after shooting and judging it as a failure.

In the foregoing description, the detection result of the motion detecting device108is used as a condition for the switching device105to output the A frame or the B′ frame, but it is also possible to determine a state change of the input signal from the operation unit107and to perform switching in response to the result of that determination. Specifically, the operation unit107has a shutter button and it is possible that the A frame is always outputted until the user half-presses the shutter button (a predetermined operation), then outputs the B′ frame when the shutter button is half-pressed.

Furthermore, a case is also envisaged where the number of capture frames of A frames and B frames in the image capturing unit102is variable within a specific time, for example within one second, in response to a detection result of the motion detecting device108. In a case where A frames are selected with priority as the output of the switching device105due to the detection result of the motion detecting device108, the image capturing unit102is controlled so that more A frames are captured compared to B frames. Due to this, the display update rate of A frames increases in the display unit106and the tracking characteristics of the display unit106are improved with respect to motion of the camera or the object.

FIG. 4Ais an example of switching control at normal times andFIG. 4Bis an example of switching control in a case where A frames are selected with priority. InFIG. 4B, it is better for the B frames to be not too separated temporally since these are multiple images for composing. This is because not being separated temporally means less positional shifts of the object between images.

Conversely, in a case where B′ frames are selected with priority as the output of the switching device105due to the detection result of the motion detecting device108, the image capturing unit102is controlled so that more B frames are captured or so that only B frames are captured. Due to this, the display update rate of B frames increases in the display unit106and the composite image update rate increases, and thus the tracking characteristics of the display unit106are improved with respect to motion of the camera or the object.

Other Embodiments

This application claims the benefit of Japanese Patent Application No. 2011-105655, filed May 10, 2011, which is hereby incorporated by reference herein in its entirety.