Image apparatus, image display apparatus and image display method

An image apparatus, comprising a image capture device that captures an image, a display device that displays the image, a display control block that enlarges and displays a first area, included in a first image, on the display device, a target setting block that sets an object in the first area as a target for tracking and a detecting block that detects a second area, including the object, in a second image which is specified to be displayed next on the display device in place of the first image, the display control block enlarging and displaying the second area on the display device.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority to and incorporates by reference the entire contents of Japanese Patent Application No. 2011-099911 filed in Japan on Apr. 27, 2011.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image apparatus, an image display apparatus and an image display method.

2. Description of the Related Art

Conventionally, a digital still camera (such as an image apparatus) records an image, captured by an imaging device having about ten million pixels, at about 30 fps to 60 fps (frames per second) with a full number of pixels. This is possible because imaging devices and the corresponding control technology has improved. If the number of recorded pixels is reduced, the image devices record images with a high frame rate, for example 1,200 fps. Some digital still cameras have a continuous shooting function, which allows the digital still camera to record plural images at a high frame rate continuously while the user keeps a release button pressed. This function is suitable for recording an image in the case of a fast moving object (such as a car and train) or an object moving unpredictably (such as a child and animal) to prevent the user from mistiming the photograph.

Plural images captured by the continuous shooting function are usually merged into one image file and saved in a memory card. Japanese laid open 2008-167067 discloses the technology in which plural images are merged and saved into one image file. A file format including plural images is defined by the Multi-Picture Format standard issued by Camera & Imaging Products Association in February 2009.

When the user has recorded plural images by the continuous shooting function, the user may wish to read out the image file and confirm the image, frame by frame, on the screen in order to select the best image. Hereinafter, the operation by which the plural images are displayed frame by frame is called frame-by-frame-forward-(backward). By the display, the user confirms the degree of focus and the expression of a face as being an object of interest (e.g. the face of child or animal) which is enlarged and displayed on the screen. In this manner, when a part of the image is enlarged and displayed on the screen, the user operation of reviewing frame-by-frame-forward-(backward) typically means that the position of the enlarged part on the screen and the magnification ratio to enlarge are fixed for the next (previous) images.

As described above, the continuous shooting function is used to record a moving object. Therefore, the position of the moving object may change frame by frame. As a result, when a part of an image is enlarged and displayed on the screen, if the user operates frame-by-frame-forward-(backward), the moving object may no longer be in the enlarged part of the screen in the next (previous) frames. Thus, in order to see the object, the user has to modify the position manually. This becomes cumbersome and problematic. Further, this problem is not limited only to the case of continuous shooting function. A similar problem will occur in the case of plural images recorded independently.

One purpose of the present embodiments of the invention is to provide an image apparatus, image display apparatus and image display method which are able to solve the above noted problem. In other words, as a result of the present improvements, when the user is confirming plural images with a part of the images being enlarged and displayed on the screen, the user is able to confirm the plural images efficiently without cumbersome operation.

SUMMARY OF THE INVENTION

An image apparatus of the present embodiments of the invention comprises an image capture device that captures an image, a display device that displays the image, a display control block that enlarges and displays a first area included in a first image on the display device, a target setting block that sets an object in the first area as a target for tracking and a detecting block that detects a second area, including the object, in a second image which is specified to be displayed next on the display device in place of the first image, wherein the display control block enlarges and displays the second area on the display device.

A display apparatus of the present embodiments of the invention comprises a display device that displays an image, a display control block that enlarges and displays a first area included in a first image on the display device, a target setting block that sets an object in the first area as a target for tracking and a detecting block that detects a second area, including the object, in a second image which is specified to be displayed next on the display device in place of the first image, wherein the display control block enlarges and displays the second area on the display device.

A displaying method of the present embodiments of the invention is performed by a display apparatus including a display device that displays an image. The method includes enlarging and displaying a first area included in a first image on the display device, setting an object in the first area as a target for tracking, detecting a second area, including the object, in a second image which is specified to be displayed next on the display device in place of the first image and enlarging and displaying the second area on the display device.

According to the present embodiments of the invention, when the first area included in the first image is enlarged and displayed on the display device, the object in the first area is set as the target for tracking and the second area including the object in the second image is enlarged and displayed in place of the first image. Thus, the user is able to confirm plural images efficiently without cumbersome operation.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Here, the best mode of the present embodiments of the invention (image apparatus, image display apparatus and image display method) is explained referring to the figures. An example applying the present embodiments of the invention to a digital still camera (digital camera) is described below.

First Embodiment

FIGS. 1A to 1Care diagrams illustrating the outward appearance of the digital camera according to the first embodiment.FIG. 1Ais a top view illustrating the digital camera,FIG. 1Bis a front view illustrating the digital camera, andFIG. 1Cis a rear view illustrating the digital camera.

The digital camera of the first embodiment includes a sub LCD1, a release shutter button2and a mode dial3on the top surface. The sub LCD displays a number of images available to be recorded. The release shutter button2is pushed in order to take a picture. The mode dial3selects a mode from a plurality of modes including a recording mode for recording images, a replay mode for replaying images and a setting mode for setting up the digital camera.

The digital camera of the first embodiment includes an AFLED (auto focus LED)10which emits light in response to an auto focus operation, a flash light LED11which emits light in response to a flash light operation, an LCD monitor12(display means) and an optical finder (rear)13on the rear surface. The LCD monitor12displays a setting view, a replayed image and a live view image (used as an electric finder).

The digital camera of the first embodiment includes a zoom button (WIDE)14which is operated in a zoom operation (WIDE), a zoom button (TELE)15which is operated in a zoom operation (TELE), a power switch16, a self timer switch17which is operated in a self timer operation, a menu switch18which is operated in a setting menu, an OK switch19which is operated in accepting a selected item, a left/image confirm switch20, a right switch21, a down/macro switch22, an up/flash light switch23and a display switch24. The left/image confirm switch20is operated to confirm a recorded image on the LCD monitor12and to move the cursor to the left on the LCD monitor12. The right switch21is operated to move the cursor to the right on the LCD monitor12. The down/macro switch22is operated to record in macro mode and to move the cursor down on the LCD monitor12. The up/flash light switch23is operated to change the flash light mode and to move the cursor up on the LCD monitor12. The display switch24is operated to change display of the LCD monitor12.

The LCD monitor12displays a mode setting display in response to the operation of the menu switch18in the digital camera. A continuous shooting mode is selected from the mode setting display on the LCD monitor12using the left/image confirm switch20, the right switch21, the down/macro switch22and/or the up/flash light switch23. The continuous shooting mode is set by operating the OK switch23. In the continuous shooting mode, plural images are captured continuously according to a single operation of the release shutter button2. This corresponds to the continuous shooting function described above.

In replay mode, set by the mode dial3, an image saved in a memory card80is displayed on the LCD monitor12. In replay mode, the displayed image is enlarged with an predetermined magnification ratio by operation of the zoom button (WIDE)14and the zoom button (TELE)15. While the enlarged image is displayed on the LCD monitor12, the enlarged position is set by operation of the left/image confirm switch20, the right switch21, the down/macro switch22and/or the up/flash light switch23.

Further, while the enlarged image is displayed on the LCD monitor12, an enlarge-and-track-frame-by-frame-forward-(backward) mode (described below) is set by keeping the display switch24pushed for a period. In the enlarge-and-track-frame-by-frame-forward-(backward) mode, an frame-by-frame-forward operation is performed by operation of the right switch21and an frame-by-frame-backward operation is performed by operation of the left/image confirm switch20. When the display switch24is pushed for a period, in the enlarge-and-track-frame-by-frame-forward-(backward) mode, then an enlarge display mode is set.

FIG. 2is a block diagram illustrating a control system of the digital still camera. The control system of the digital camera includes a motor driver28, a CMOS sensor30which is an image sensor, an AFE (analog front end)40, a TG (timing generator)50and a signal processor IC60. The motor driver28drives a zoom lens25, a focus lens26and a mechanical shutter27. The AFE40generates digital image data from analog image data output by the CMOS sensor30. The TG50outputs a timing signal to the CMOS sensor30and the AFE40. The signal processor IC60performs digital signal processing on the digital image data from the AFE40and controls operation of the digital camera. The signal processor IC60is connected to the LCD monitor12, a DRAM70, the memory card80, a ROM90and an operation unit100.

The mechanical shutter27is a shutter operated mechanically. Light is incident on the CMOS sensor30when the mechanical shutter27is opened and light is blocked when the mechanical shutter27is closed. The motor driver28drives the zoom lens25, the focus lens26which is an optical lens, and the mechanical shutter27. A CPU61included in the signal processor IC60controls the motor driver28which drives the zoom lens25, the focus lens26, and the mechanical shutter27.

The CMOS sensor30is a solid state image sensor having a conversion function which converts optical light to electrical signals. The CMOS sensor30outputs an electric signal proportional to the amount of light, when an object is imaged by the optical lens. The CMOS sensor30is driven by a timing signal supplied by the TG50. Exposure control and read out control on each pixel is performed according to the timing signal. A signal from the CMOS sensor30is input to the AFE40.

The AFE40includes a CDS (correlated double sampling)41, an AGC (auto gain control)42, and an A/D (analog to digital) converter43. The CDS41performs sampling and holding (correlated double sampling) on analog image data output from the CMOS sensor30. The AGC performs gain control on a signal sampled by the CDS41. The A/D converter43converts a signal output from the AGC42to a digital image signal. Each part included in the AFE40is driven by a timing signal supplied by the TG50in synchronization with the CMOS sensor30. Gain of the AGC42is set according to an order by the CPU61in the signal processor IC60. The digital image signal output from the AFE40is input to the signal processor IC60.

The TG50receives a VD signal (vertical synchronizing signal) and an HD signal (horizontal synchronizing signal) which are generated by a synchronizing signal generator69included in the signal processor IC60. The TG50generates a timing signal for driving the CMOS sensor30and the AFE40according to the HD signal and the VD signal. The timing signal is input to the CMOS sensor30and the AFE40.

The signal processor IC60includes the CPU61, a CMOSI/F block62, a memory controller63, a YUV conversion block64, an encoder/decoder65, a display output controller66, a resize processing block67, a media I/F block68and the synchronizing signal generator69. The CMOSI/F block62reads out digital image data from the AFE40in synchronization with the VD signal and the HD signal. The memory controller63controls write and read out from a SDRAM70. The YUV conversion block64converts a digital image signal input from the CMOSI/F block62to YUV data for displaying and recording. The encoder/decoder65performs encode and decode processing in a certain file format such as JPEG. The display output controller66controls outputting of the image data to the LCD monitor12and to other external display devices. The resize processing block67changes the size of the image data corresponding to the size for displaying and recording. The media I/F block68controls write and read out from the memory card80. The synchronizing signal generator69generates the VD signal and the HD signal.

The digital image data output from the AFE40is taken into the signal processor IC60via the CMOSI/F62per frame and image correction, such as black level, detective pixel, and shading, is performed by the signal processor IC60. Then, the digital image data is saved as RGB-RAW image data in the SDRAM70.

The RAW-RGB image data is read out by the memory controller63, image correction, such as gain multiplication of white balance, gamma correction and RGB interpolating correction, is performed and the RAW-RGB image data is provided to the YUV conversion block64. The YUV conversion block64performs image correction such as edge enhancement and color setting and converts the RAW-RGB image data to YUV image data (YCbCr). The YUV image data is saved in the SDRAM70.

The YUV image data is read out by the memory controller63and is provided to the display output controller66with OSD (on screen display) data which displays an icon indicating the operation mode. The display output controller66combines the YUV image data and the OSD data and generates display data. The display output controller66combines the display data with a synchronization signal and provides the combination to the LCD monitor12. The display output controller66provides the display signal as a TV video signal for an external display device. In case the external display device is a TV in the NTSC system, the resize processing block67adjusts the size horizontally and vertically and provides the adjusted signal to the TV.

Recording data stored in the memory card80is now described. The YUV image data is read out by the memory controller63and is provided to the encoder/decoder65based on a command from the CPU61. The encoder/decoder65generates compressed data such as JPEG. The YUV image data, saved in the SDRAM70, corresponds to the full size of the CMOS sensor30. If the size of the image data to be saved in the memory card80is smaller than the size of the image data produced by the CMOS sensor30, the resize processing block67reduces the size for recording and provides the reduced data to the encoder/decoder65. The compressed data is saved in the SDRAM70and recorded in the memory card80via the media I/F block after adding a header. When plural images, obtained by continuous shooting mode, are recorded in the memory card80, the plural images may be recorded as one file such as a file of the Multi-Picture Format standard.

When an image saved in the memory card80is replayed, the compressed data, saved in the memory card89, is read out via the media I/F block68and is saved in the SDRAM70based on a command from the CPU61. Then, the compressed data is provided to the encoder/decoder65. The YUV image data decoded by the encoder/decoder65is provided to the display output controller66. The display output controller66generates display data and outputs the display data to the LCD monitor12. In this manner, an image is displayed (replayed) on the LCD monitor12. When an enlarged image is displayed on the LCD monitor12, the display output controller66outputs display data of the enlarged image to the LCD monitor12based on a command from the CPU61. In this manner, the enlarged image is displayed.

Each part included in the signal processor IC60performs processing according to the CPU61. For example, the CPU61loads a control program and control data from the SDRAM70. The control program and control data is saved in the ROM90, for example, and is used to control the digital camera. The CPU61controls the whole operation of the digital camera according to the control program and control data. In other words, the CPU61controls the motor driver28, the AFE40, the TG50and the other parts of the signal processor IC60so that a proper image is captured, recorded and displayed according to the direction from the operation unit100, direction from the remote controller and so on. The operation unit100accepts input from the user and includes buttons and switches such as the release shutter button2, the zoom buttons14,15, the menu switch18, the right switch21, the left/image confirm switch20, the display switch24and so on.

The digital camera of the present embodiment performs enlarge-and-track-frame-by-frame-forward-(backward). The enlarge-and-track-frame-by-frame-forward-(backward) is defined by the procedure as shown below.

When plural images from continuous shooting are read out the memory card80and are displayed on the LCD monitor12frame-by-frame, an object, included in part of an image enlarged and displayed, is set as a target for tracking. In the case of performing enlarge-and-track-frame-by-frame-forward, an area including the object, set as the target for tracking, is detected from a next image (later in time) to be displayed, and the area, in the next image (later in time), is displayed on the LCD monitor12. In the case of performing enlarge-and-track-frame-by-frame-backward, an area including the object, set as the target for tracking, is detected from a next image (previous in time) to be displayed, and the area, in the next image (previous in time), is displayed on the LCD monitor12.

FIG. 3is a functional block diagram illustrating the CPU61which performs the enlarge-and-track-frame-by-frame-forward-(backward) operation. The CPU61implements the function of a target setting block101, a detecting block102and a display control block103by executing the control program stored in the ROM90. In the present embodiment, the CPU61implements these functions by executing the control program. Alternatively, specific hardware such as an ASIC (application specific integrated circuit) or a FPGA (field-programmable gate array) may be substituted for the control program to realize some or all of these functions.

When the first part of the first image is enlarged and displayed on the LCD monitor12and the user keeps the display switch24of the operation unit100pushed for a period, the target setting block101sets an object included in the first part displayed on the LCD monitor12as the target for tracking. Tracking is defined as a process to track an object set as the target among different plural images. Various methods of tracking are available in the present embodiment.

Tracking technology is generally used in an area for AF (auto focus) and AE (auto exposure) for an object to be photographed. The present embodiment uses tracking technology to perform enlarge-and-track-frame-by-frame-forward-(backward) when the image is enlarged and displayed. In other words, in the case of performing enlarge-and-track-frame-by-frame-forward-(backward), the object is tracked among different images using tracking technology and the object is enlarged and displayed. If the digital camera of the present embodiment has the function of tracking for AF and AE (tracking object block for capturing), it is desirable to use this common function for performing enlarge-and-track-frame-by-frame-forward-(backward). Since the function of the tracking object block for capturing may also be used to enlarge-and-track-frame-by-frame-forward-(backward), the number of programs used for tracking the object is reduced and the capacity of the ROM90is conserved, resulting in the cost being reduced.

Here, for example, general template matching is used to track the object. The target setting block101extracts an amount of characteristic (characteristic of brightness or color, edge changing brightness or color dramatically or object shape identified from edge and so on) from the image enlarged and displayed on the LCD monitor12. The extracted amount of characteristic is set as the template and saved in the SDRAM70. The process described above corresponds to the target setting for tracking the object.

In the present embodiment, in order to transition to enlarge-and-track-frame-by-frame-forward-(backward), an exemplary operation is to keep the display switch24pushed for a period. Other operations for initiating the transition are alternatively available. In the present embodiment, images are displayed frame-by-frame as the user operates the right switch21and the left/image confirm switch20. Alternatively, the digital camera changes images frame-by-frame automatically according to a predetermined time without the user's operation.

The detecting block102detects an area, including the target, from a second image which is displayed after the first image currently displayed according to the user's operation. The target is set by the target setting block101.

When the enlarge-and-track-frame-by-frame-forward-(backward) operation is available, and the user operates the right switch21(frame-by-frame-forward operation), the detecting block102specifies the image which is captured after the first image in time, as an image to be displayed next. Further, when the enlarge-and-track-frame-by-frame-forward-(backward) operation is available, and the user operates the left/image confirm switch20(frame-by-frame-backward operation), the detecting block102specifies the second image which is captured before the first image in time, as an image to be displayed next. The detecting block102reads out the template saved in the SDRAM70and detects the area including an amount of characteristic similar to the amount of characteristic in the template for the area including the object for tracking.

The display control block103processes the enlarging and displaying part of the image displayed in the LCD monitor12. Specifically, the user is able to operate the operation unit100(the zoom button (WIDE)14, the zoom button (TELE)15, the left/confirm switch20, the down/macro switch22and the up/flash light switch23etc.) affecting the images displayed on the LCD monitor12in replay mode. The display control block103specifies coordinate information showing the position and enlargement factor of the part set by the user's operation and outputs an enlarging and displaying command to the display output controller66. The display controller66outputs the display data of the part specified by the user to the LCD monitor12. The part specified by the user is enlarged and displayed on the LCD monitor12.

The user operates the operation unit100using the right switch21(frame-by-frame-forward) and the left/confirm switch20(frame-by-frame-backward) in order to implement the enlarge-and-track-frame-by-frame-forward-(backward) operation. If the detecting block102detects an area including the target from the next displayed image, the display control block103performs a process for enlarging and displaying the detected area on the LCD monitor12. Specifically, the display control block103specifies the coordinate information showing the position and enlargement factor of the area detected by the detecting block102in order to enlarge and display the area detected by the detecting block102. The display control block103outputs an enlarging and displaying command to the display output controller66. The display output controller66outputs display data of the area detected by the detecting block102to the LCD monitor12. The area detected by the detecting block102is then enlarged and displayed on the LCD monitor12.

During the enlarge-and-track-frame-by-frame-forward-(backward)-operation, unless the display switch24is kept pushed (exiting the mode), the detecting block102and the display control block103perform processing described above over and over according to the operation of the right switch21and the left/confirm switch20. Therefore, the enlarge-and-track-frame-by-frame-forward-(backward) operation, which processes the enlarging and displaying area, including the object set as the target for tracking among the plural images, can be realized. According to the enlarge-and-track-frame-by-frame-forward-(backward) operation, since the area of interest, which includes the object, is enlarged and displayed according to the operation of enlarge-and-track-frame-by-frame-forward-(backward), the user does not have to perform a cumbersome operation. For instance, the user does not have to perform an operation such as correcting the enlarged position manually and the user can confirm the degree of focus regarding the object or the expression of the object efficiently.

FIG. 4is a flowchart illustrating a process flow for the CPU61in the performance of the enlarge-and-track-frame-by-frame-forward-(backward) operation. The process flow shown inFIG. 4starts when the user keeps the display switch24pushed for a predetermined period when part of the image is enlarged and displayed on the LCD monitor12.

The target setting block101of the CPU61sets an object as a target for tracking, when the object is included in the area currently enlarged and displayed on the LCD monitor12(step S101). Specifically, the target setting block101extracts the amount of characteristic of the image from the area currently enlarged and displayed on the LCD monitor12. The target setting block101saves the extracted amount of characteristic as a template in the SDRAM70. When step S101finishes, the process proceeds to step S102.

The CPU61checks whether the user operates the operation unit100(step S102). If the user performs some operation (step S102YES), the flow proceeds to step S103. If the user does not perform any operation (step S102NO), the CPU61continues checking.

The CPU61checks whether the user's operation is to keep the display switch24pushed (step S103) for a predetermined period of time. If the user's operation is not to keep the display switch24pushed (step S103NO), the flow proceeds to step S104. If the user's operation is to keep the display switch24pushed (step S103YES), the process flow finishes inFIG. 4. When the process flow finishes, the mode of the digital camera returns to normal enlarge and display mode.

The CPU61checks whether the user operates the right switch21(step S104). If the user operates the right switch21(step S104YES), the flow proceeds to step S105. If the user does not operate the right switch21(step S104NO), the flow proceeds to step S106.

The detecting block102in the CPU61specifies a later in time image than the current image, whose part is enlarged and displayed on the LCD monitor12, as the next displayed image (step S105). When the step S105finishes, the flow proceeds to step S108.

The CPU61checks whether the user operates the left/image confirm switch20(step S106). If the user operates the left/image confirm switch20(step S106YES), the flow proceeds to step S107. If the user does not operate the left/image confirm switch20(step S106NO), the flow proceeds to step S102since the operation is not valid with respect to the enlarge-and-track-frame-by-frame-forward-(backward) operation.

The detecting block102in the CPU61specifies a previous in time image, than the current image, whose part is enlarged and displayed on the LCD monitor12, as the next displayed image (step S107). When the step S107finishes, the flow proceeds to step S108.

The detecting block102detects the area including the object set as the target, in step S101, from the next displayed image. Specifically, the detecting block102reads out the template saved in the SDRAM70by the target setting block101. The detecting block102detects the area that includes the amount of characteristic which is similar to the amount of characteristic of the template and sets the detected area as the area including the object for tracking in the next displayed image (step S108). When the step S108finishes, the process flow proceeds to step S109.

The display control block103in the CPU61determines whether to detect the area, including the object for tracking, from the next displayed image in step S105or S107(step S109). If the area is detected (step S109YES), the flow proceeds to step S110. When the area is not detected (step S109NO), the flow proceeds to step S111.

The display control block103outputs a command, which enlarges and displays the area detected in step S109on the LCD monitor12, to the display controller66(step S110). The command includes coordinate information and an enlargement factor for the area being enlarged and displayed as the next displayed image. The enlargement factor is determined so that the size of the object in the current displayed image is the same as the size of the object in the next displayed image. For example, the enlargement factor is proportionally determined according to the size of the object (tracked target) in the image, which is currently enlarged and displayed on the LCD monitor12, and to the size of the object in the next displayed image. In step S110, the display of the LCD monitor12is updated and the area including the object (tracked target), in the next displayed image, is enlarged and displayed on the LCD monitor12. When step S110finishes, the flow returns to step S102.

In step S111, The display control block103outputs the command not to change the coordinate information and the enlargement factor of the area being enlarged and displayed, to the display output controller66, when the next displayed image, specified in step S105or S107, is displayed. In other words, the command includes coordinate information and an enlargement factor, which are the same as the coordinate information and enlargement factor of the enlarged and displayed area, for the image currently enlarged and displayed on the LCD monitor12. According to step S111, the display of the LCD monitor12is updated and the same area as the current image is enlarged and displayed on the LCD monitor12for the next image. When the step S111finishes, the flow returns to step S102.

In the above description, when the user operates the right switch21(frame-by-frame forward), the displayed image is switched from current displayed image to the next image. For example, in the case of plural recorded images included in one file, such as Multi-Picture Format, the images are displayed frame-by-frame, when the user operates the right switch21(frame-by-frame forward) the images are switched. If the currently displayed image is the last image in the file, the next displayed image may be the first image in the file.

Next, referring toFIGS. 5 to 8, transition of the enlarged and displayed image on the LCD monitor12is explained with respect to enlarge-and-track-frame-by-frame-forward-(backward). Here,FIGS. 5 to 8show plural images of a running dog obtained by continuous shooting mode. In this example, the user confirms the dog's face in each image.

For comparison,FIG. 5shows plural images that are enlarged and displayed frame-by-frame on the LCD monitor12by conventional enlarged-frame-by-frame-forward.FIG. 5Ashows the whole area of the plural images. The area surrounded by the dashed line inFIG. 5Ais enlarged and displayed on the LCD monitor12.FIG. 5Bshows a transition between the plural images enlarged and displayed on the LCD monitor12.

As is shown inFIG. 5A, the area enlarged and displayed on the LCD monitor12is fixed with respect to the whole image according to conventional enlarged-frame-by-frame-forward. Even though the enlarged and displayed area (dashed line) shows the dog's face in the first image, the enlarged and displayed area moves from the dog's face to the dog's body in the second image and moves from the dog's body to the dog's tail in third image. In other words, the enlarged and displayed area moves from the dog's face to the dog's tail when proceeding frame-by-frame-forward. Although the dog's face is enlarged and displayed on the LCD monitor12in the first image, the enlarged and displayed area on the LCD display12moves from the dog's body to the dog's tail when proceeding frame-by-frame-forward. When performing frame-by-frame-backward, the display order is inverted and the result is similar to the frame-by-frame-forward case.

In conventional technology, the user has to modify the position of the enlarged and displayed area in the second and third image in order to confirm the dog's face. This operation may become extremely cumbersome.

FIG. 6shows plural images that are enlarged and displayed frame-by-frame on the LCD monitor12by enlarge-and-track-frame-by-frame-forward according to the digital camera of the present embodiment.FIG. 6Ashows the whole area of each of the plural images. The area surrounded by the dashed line inFIG. 6Ais enlarged and displayed on the LCD monitor12.FIG. 6Bshows the transition between the plural images enlarged and displayed on the LCD monitor12.

When the enlarge-and-track-frame-by-frame-forward-(backward) operation of the present embodiment is available, an object included in the area enlarged and displayed on the LCD monitor12is set as the tracked target. The area, including the object set as the target, is detected in next image (later or earlier in time) and the detected area is enlarged and displayed on the LCD monitor12.

For example, the user may enlarge and display a dog's face in the first image ofFIG. 6A. When the dog's face is enlarged and displayed on the LCD monitor12in the first image ofFIG. 6B, and the user keeps the display switch24pushed down for a predetermined period, the enlarge-and-track-frame-by-frame-forward-(backward) operation is executed and the dog's face is set as the target tracking object. When performing frame-by-frame-forward (operation of the right switch21), the area including the dog's face in the second image ofFIG. 6Ais detected and the detected area including the dog's face is enlarged and displayed on the LCD monitor12shown as the second image inFIG. 6B. When performing further frame-by-frame-forward (operation of the right switch21), the area including the dog's face in the third image ofFIG. 6Ais detected and the detected area including the dog's face is enlarged and displayed on the LCD monitor12shown as the third image inFIG. 6B. When performing the frame-by-frame-backward operation (operation of the left/image confirm switch21), the displaying order is inverted and the result is similar to the frame-by-frame-forward operation (the operation of the right switch20).

The digital camera of the present embodiment enlarges and displays plural images in series on the LCD monitor12by the enlarge-and-track-frame-by-frame-forward-(backward) operation.

The area including the object of interest (described as the dog's face in the example), among the plural images, is enlarged and displayed on the LCD monitor12without the need for a cumbersome operation. The user can confirm the plural images in order to select the best shot among the plural images effectively, quickly, and easily.

In the example shown inFIG. 6, the size of the object of interest (dog's face) does not substantially change among the plural displayed images. On the other hand, the size of an object may change among the plural images depending on how the object moves. The size of the object enlarged and displayed on the LCD monitor12becomes substantially the same among the plural images by determining an enlargement factor according to the size of the object in the whole image.

In the case ofFIG. 7, when the size of the object of interest(dog's face) changes among the plural displayed images, the size of the object enlarged and displayed on the LCD monitor12becomes substantially the same among the plural images.FIG. 7Ashows the whole area of each of the plural images. The area surrounded by the dashed line inFIG. 7Ais enlarged and displayed on the LCD monitor12.FIG. 7Bshows the transition between the plural images enlarged and displayed on the LCD monitor12.

For example, the user may enlarge and display a dog's face in the first image ofFIG. 7A. When the dog's face is enlarged and displayed on the LCD monitor12in the first image ofFIG. 7B, and the user keeps the display switch24pushed in for a predetermined period, the enlarge-and-track-frame-by-frame-forward-(backward) operation is executed and the dog's face is set as the target tracking object. When performing frame-by-frame-forward (operation of the right switch21), the area including the dog's face in the second image ofFIG. 7Ais detected. The size of the dog's face in the second image ofFIG. 7Ais larger than the size of the dog's face in the first image ofFIG. 7A. The enlargement factor for the area including the dog's face in the second image is smaller than the enlargement factor for the area including the dog's face in the first image taking into account the ratio of the size of the dog's face in the first image and the size of the dog's face in the second image. Therefore, the size of the dog's face, enlarged and displayed on the LCD monitor12, is set to be substantially the same in the first image and the second image.

As shown inFIG. 6, the object of interest (dog's face) is detected on three images in the series. However, the object of interest may temporarily exit the frame in the case of continuous shooting and a fast moving object. When the object of interest is not detected among any of the plural images, the area enlarged and displayed remains the same as the enlarged and displayed area of the former displayed image. Thereafter, when the object of interest is detected in the next displayed image by the frame-by-frame-forward-(backward) operation, the detected area is enlarged and displayed. Therefore, even though the object of interest is missed in some of the images, the tracking of the object is not ended and the user does not need to reset the object as the target again. This reduces the difficultly of the operation.

FIG. 8shows the enlarged and displayed images in series on the LCD monitor12using the enlarge-and-track-frame-by-frame-forward-(backward) operation. Further,FIG. 8illustrates the example of the object of interest temporarily exiting the frame and returning into the frame between the plural images.FIG. 8Ashows the whole area of each of the plural images. The area surrounded by a dashed line inFIG. 8Ais enlarged and displayed on the LCD monitor12.FIG. 88shows the transition between the plural images enlarged and displayed on the LCD monitor12.

For example, the user enlarges and displays the dog's face in the first image ofFIG. 8A. When the dog's face is enlarged and displayed on the LCD monitor12in the first image ofFIG. 8B, and the user keeps the display switch24pushed in for a period, the enlarge-and-track-frame-by-frame-forward-(backward) operation is executed and the dog's face is set as the target for tracking. When performing frame-by-frame-forward (operation of the right switch21), and the area including the dog's face in the second image ofFIG. 8Ais not detected despite the detecting process, the same area as the first image inFIG. 8Ais enlarged and displayed. Therefore, as is shown in the second image ofFIG. 8B, the dog's tail is temporarily enlarged and displayed on the LCD monitor12.

Thereafter, when performing frame-by-frame-forward (operation of the right switch21), the dog's face is detected in the third image ofFIG. 8A. Since the dog's face is detected in the third image ofFIG. 8A, the area including the detected dog's face is enlarged and displayed on the LCD monitor12. Thus, the dog's face is again enlarged and displayed on the LCD monitor12as shown in third image ofFIG. 8B. When performing the frame-by-frame-backward operation (operation of the left/image confirm switch21), the display order is inverted but the result is similar to the frame-by-frame-forward operation (the operation of the right switch20).

Second Embodiment

The digital camera of the present embodiment includes the tracking object block, implemented by a processor, used for capturing and which has a tracking function for performing AF (auto focus) and AE (auto exposure) on an object to be photographed. The digital camera of present embodiment uses information to track the object when plural images, obtained by continuous shooting, are enlarged and displayed frame-by-frame. As a result, the process of enlarge-and-track-frame-by-frame-forward-(backward) becomes faster. The basic composition and operation of the digital camera of the present embodiment are the same as those of first embodiment. The characteristic of the present embodiment is described below with common explanations being omitted.

The image data, by which the tracking object block tracks the object, is recorded with information enabling common tracking of the object. Specifically, metadata such as Exif (Exchangeable Image File) data, which includes information enabling tracking of the object by the tracking object block, can be added to the image data. As described above, plural images obtained by continuous shooting may be saved in one file such as of the Multi-Picture Format. Exif data can be added to the image data included in the Multi-Picture Format file.

According to the present embodiment, the Exif data includes coordinate information for the area including the object tracked by the tracking object block. The information added to image data may be other information specifying the area including object tracked by the tracking object block. For example, an image may be divided into plural blocks in a predetermined manner. Instead of coordinate information, information specifying the block including the object tracked by the tracking object block can be added.

Similar to the first embodiment, when the user keeps the display switch24pushed for a period while part of the first image is enlarged and displayed on the LCD monitor12, the target setting block101in the CPU61sets an object included in the part of first image, enlarged and displayed on the LCD monitor12, as the target for the enlarge-and-track-frame-by-frame-forward-(backward) operation. If the currently displayed image is an image obtained by continuous shooting, the CPU61checks whether the object set as the target for enlarge-and-track-frame-by-frame-forward-(backward) operation corresponds to an object tracked by the tracking object block.

Specifically, the CPU61compares the coordinates for the part of the first image enlarged and displayed on the LCD monitor12with the coordinate information included in the Exif data of the first image (the coordinate information for the area including the object tracked by the tracking object block). The CPU61checks whether the part of the first image enlarged and displayed on the LCD monitor12corresponds to the area including the object tracked by the tracking object block. The phrase “corresponds to” means not only corresponding perfectly but also corresponding in a way such that both areas include some portion of the object. For example, if certain percentage (such as 80%) of the part of first image enlarged and displayed on the LCD monitor12overlaps the area including the object tracked by the tracking object block, the CPU61concludes that the areas correspond to each other.

When the part of the first image enlarged and displayed on the LCD monitor12corresponds to the area including object tracked by the tracking object block, the object tracking is simplified in the enlarge-and-track-frame-by-frame-forward-(backward) operation. When performing the frame-by-frame-forward-(backward) operation in the case that the enlarge-and-track-frame-by-frame-forward-(backward) operation is available, the display control block103obtains coordinate information from the Exif data included in the next displayed image data. This obtaining process is performed without the detecting block102detecting the area, including the object set as the target, from the next displayed image data. Once the coordinate information is obtained, the area corresponding to the coordinate information is enlarged and displayed on the LCD monitor12. In other words, when the object set as the target for the enlarge-and-track-frame-by-frame-forward-(backward) operation corresponds to the object tracked by the tracking object block, the digital camera of present embodiment utilizes information available at the time of capturing without performing a detecting process in the enlarge-and-track-frame-by-frame-forward-(backward) operation. As a result, the process becomes simpler and faster.

With respect to the digital camera of the present embodiment,FIGS. 9A&Bare a flowchart illustrating a process flow for the CPU61performing the enlarge-and-track-frame -by-frame-forward-(backward) operation. The process flow shown inFIG. 9Astarts when the user keeps the display switch24pushed in for a period when part of the image is enlarged and displayed on the LCD monitor12, in the same manner as is described with respect toFIG. 4.

The target setting block101of the CPU61sets an object as the target for tracking, from an object included in the area currently enlarged and displayed on the LCD monitor12(step S201), in the same manner as step S101inFIG. 4. When the step S201finishes, the flow proceeds to step S202.

The CPU61checks whether an image included in the area enlarged and displayed on the LCD monitor12is one of the images captured by continuous shooting (step S202). For example, the CPU61checks whether the image is taken from a file of the Multi-Picture Format. If the image, including the area enlarged and displayed on the LCD monitor12, is one of the images captured by continuous shooting (step S202YES), the flow proceeds to step S203. If the image is captured individually (step S202NO), the flow proceeds to step S206.

The CPU61checks whether the Exif data of the image, including the area enlarged and displayed on the LCD monitor12, includes coordinate information for an area including the object tracked by the tracking object block (step S203). If the Exif data includes the coordinate information (step S203YES), the flow proceeds to step S204. If the Exif data does not include coordinate information (step S204NO), the flow proceeds to step S206.

The CPU61checks whether the area currently enlarged and displayed on the LCD monitor12corresponds to the area including the object tracked by the tracking object block (step S204). If both areas correspond to each other (step S204YES), the flow proceeds to step S205. If not (step S204NO), the flow proceeds goes to step S206.

In step S205, the CPU61sets a simple process flag to1. The simple process flag is 1-bit data and indicates that the tracking object is being performed simply as described above in the enlarge-and-track-frame-by-frame-forward-(backward) operation. In particular, when the simple process flag is1, the simple tracking process is performed. When the simple process flag is0, the simple tracking process is not performed. When step S205finishes, the flow proceeds to S207.

In step S206, the CPU61sets the simple process flag to0. When step S206finishes, the flow proceeds to S207.

As in step S102, the CPU61checks whether the user operates the operation unit100(step S207). If the user performs some operation (step S207YES), the flow proceeds to step S208. If the user does not perform any operation (step S207NO), the CPU61continues checking.

As in step S103, the CPU61checks whether the user's operation is to keep the display switch24pushed in for a period (step S208). If the user's operation is not to keep the display switch24pushed (step S208NO), the flow proceeds to step S209. If the user's operation is to keep the display switch24pushed for a period (step S208YES), the process flow finishes inFIG. 9B. When the process flow finishes, the mode of the digital camera returns to normal enlarge and display mode.

As in step S104, the CPU61checks whether the user operates the right switch21(step S209). If the user operates the right switch21(step S209YES), the flow proceeds to step S210. If the user does not operate the right switch21(step S209NO), the flow proceeds to step S211.

As in step S105, the detecting block102in the CPU61specifies a later in time image, from the current image having a part which is enlarged and displayed on the LCD monitor12, as the next displayed image (step S210). When step S210finishes, the flow proceeds to step S213.

As in step S106, the CPU61checks whether the user operates the left/image confirm switch20(step S211). If the user operates the left/image confirm switch20(step S211YES), the flow proceeds to step S212. If the user does not operate the left/image confirm switch20(step S211NO), the flow proceeds to step S207since the operation is not valid with respect to the enlarge-and-track-frame-by-frame-forward-(backward) operation.

As in step S107, the detecting block102in the CPU61specifies a previous in time image, from the current image having a part which is enlarged and displayed on the LCD monitor12, as the next displayed image (step S212). When the step S212finishes, the flow proceeds to step S213.

As in step S108, the detecting block102detects the area including the object set as the target in step S201from the next displayed image (step S214). When step S214finishes, the flow proceeds to step S215.

As in step S109, the display control block103in the CPU61determines whether to detect the area including the object for tracking from the next displayed image in either step S210or S212(step S215). If the area is determined to be detected (step S215YES), the flow proceeds to step S216. If the area is determined not to be detected (step S215NO), the flow proceeds to step S217.

As in step S110, the display control block103outputs a command to the display controller66that enlarges and displays the area, detected in step S215, on the LCD monitor12(step S216). According to step S216, the display of the LCD monitor12is updated and the area, including the object (tracked target) in the next displayed image, is enlarged and displayed on the LCD monitor12. When step S216finishes, the flow returns to step S207.

As shown the step S111, the display control block103outputs a command to the display output controller66keeping the coordinate information and enlargement factor of the area enlarged and displayed the same, when the next displayed image, specified in either step S210or S212, is displayed (step S217). In step S217, the display of the LCD monitor12is updated and the area enlarged and displayed remains the same as the area in the currently enlarged and displayed image on the LCD monitor12. When step S217finishes, the flow returns to step S207.

The display control block103outputs a command to the display output controller66to enlarge and display the area specified by the coordinate information in Exif data, when the next displayed image, specified in step S210or S212, is displayed (step S219). In step S219, the display of the LCD monitor12is updated and the area, including the object tracked by the tracking object block, is enlarged and displayed on the LCD monitor12. When step S219finishes, the flow returns to step S207.

In step S220, the display control block103outputs a command to the display output controller66to not change the coordinate information and the enlargement factor of the enlarged and displayed area, when the next displayed image, specified in step S210or S212, is displayed.

In step S220, the display of the LCD monitor12is updated and the area enlarged and displayed remains the same as the area currently enlarged and displayed image on the LCD monitor12. When step S220finishes, the flow returns to step S207.

When plural images, captured by continuous shooting mode, are displayed in the enlarge-and-track-frame-by-frame-forward-(backward) operation, and the plural images are captured by tracking the object and the image data of the plural images includes information specifying the tracked area (coordinate information), the digital camera of the present embodiment utilizes the information specifying tracked area. In other words, when the object set as the target in the enlarge-and-track-frame-by-frame operation is the same as the object for capturing, the simple process is performed. In the simple process, the area, specified by the information indicating the tracked area included in the next displayed image, is enlarged and displayed. The detecting process (such as template matching), that detects the target object from next displayed image, is omitted. According to the digital camera of the present embodiment, the detecting process, which has the highest processing load in the enlarge-and-track-frame-by-frame-forward-(backward) operation, can be omitted resulting in faster processing.

Although the invention has been described with respect to specific embodiments, the claims are not limited thereto but are construed as embodying all modifications and alternative constructions that may occur to one skilled in the art that fairly fall within the basic teaching herein set forth. The construction and operation of the digital camera of the present embodiments are exemplarily described and can be modified in various manners according to purpose and use.

The described embodiments describe the example of application to a digital camera including an image capturing device. Alternatively, the present embodiments may be applied to an image display apparatus which displays an image read out from recording media (e.g. memory card) or taken from another device connected directly thereto or to a communication network. The function of the target setting and detecting object is performed by a control program executed by a CPU which controls the image display apparatus or by specific hardware such as ASIC or FPGA. The image display apparatus having the function which is the same as the digital camera explained above can also be realized.