Burst image capture method and image capture system thereof

The present invention discloses an image capture method comprises capturing a sequence of image frames; providing the sequence of image frames for preview along a first path and buffering the sequence of image frames into a buffer along a second path separately and concurrently; receiving an input command via an input interface unit; and in response to the input command, extracting a portion of the sequence of image frames from the buffer and encoding the portion of image frames along the second path; and storing the portion of encoded image frames in a memory unit.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention discloses an image capture method and an image capture system thereof; more particularly, a burst image capture method of providing image frames along separate paths for preview and buffering concurrently.

2. Description of the Prior Art

As the popularity of portable devices increases hugely, functions equipped within the portable device becomes more variant. One of important functions that are widely used by consumers is image capture, such as camera. However, image capture operation of a portable device is often limited by its slow performance. When user launches the camera application, the camera of the portable device is in a preview mode and provides preview image frames of a scene on the display screen at first. When user desires to take a photo of a certain scene and triggers the shutter button (for example press shutter icon on the touch screen), the camera switches from preview mode to capture mode, and the preview of image frames would stop refreshing and the portable device is temporarily occupied by processing the captured image frame. As a result, if a user desires to capture multiple image frames, the time required to capture all image frames would be significantly long due to the camera has to switch in between preview mode and capture mode, and thus cannot provide burst photo within a short time. This is because that the camera provides preview of image frames in low resolution, but in capture mode, the image frames are captured in high resolution comparing with preview mode. Consequently, the camera needs to spend time to configure settings in response to mode change.

Another conventional way of performing burst image capture is to use the low resolution image frames in preview mode as captured image frames in capture mode. This can save the delay caused by mode change. However, the image quality may not be satisfying, and still the display screen is idled from refresh. Such kind of burst image capture generally performs image capture of predetermined number of image frames, which means that when user triggers such burst capture, the camera automatically captures the predetermined number of image frames. While the camera is capturing the image frames, the display screen is paused at the image frame previously display (prior to the trigger of burst image capture).

Therefore, there exists a need of an image capture method and system that is capable to capture high resolution image frame continuously without stopping refresh of preview image frame on display screen.

SUMMARY OF THE INVENTION

The claimed invention discloses a burst image capture method. The burst image capture method comprises capturing a first sequence of image frames; generating a second sequence of image frames according to the first sequence of image frames; providing the second sequence of image frames for preview; generating a third sequence of image frames according to the first sequence of image frames and buffering the third sequence of image frames concurrently with the displaying of the second sequence of image frames; receiving an input command; and in response to the input command, extracting a fourth sequence of image frames out of the third sequence of image frames, encoding the fourth sequence of image frames into a fifth sequence of image frames, and storing the fifth sequence of image frames.

The claimed invention discloses an image capture method. The method comprises capturing a sequence of image frames; providing the sequence of image frames for preview along a first path and buffering the sequence of image frames into a buffer along a second path separately and concurrently; receiving an input command via an input interface unit; and in response to the input command, extracting a portion of the sequence of image frames from the buffer and encoding the portion of image frames along the second path; and storing the portion of encoded image frames in a memory unit.

The claimed invention also discloses an image capture system. The image capture system comprises an image sensor module, configured to capture a first sequence of image frames; an image signal processor, configured to generate and provide a second sequence of image frames and a third sequence of image frames according to the first sequence of image frames; a touch screen, configured to display the second sequence of image frames and configured to receive an input command; a backend processing unit, configured to buffer the third sequence of image frames, and in response to the input command, extract a fourth sequence of image frames out of the third sequence of image frames and encode the fourth sequence of image frames into a fifth sequence of image frames; and a memory unit, configured to store the fifth sequence of image frames; wherein the second sequence of image frames and the third sequence of image frames are of different resolutions and provided along separate paths concurrently.

DETAILED DESCRIPTION

For overcoming the abovementioned limitations of the prior art while performing fast burst image capture, the present invention discloses a burst image capture method and an image capture system utilizing the continuous image capture method. With the aid of the burst image capture method and the image capture system of the present invention, lags between capturing multiple image frames can be minimized. In addition, a user of the image capture system would benefit from capturing image frames having high resolution by merely using simple input commands.

Please refer toFIG. 1, which illustrates an image capture system100according to one embodiment of the present invention. As shown inFIG. 1, the image capture system100includes an image sensor module110, an image signal processor120, a backend processing unit130, a touch panel140, and a memory unit150. Note that the image capture system100may be implemented in a mobile electronic device in embodiments of the present invention. The mobile electronic device may be a mobile phone, tablet, PDA, digital camera, digital camcorder, game console, and/or other suitable device, and the image capture system100may be implemented as a camera function with burst mode capture.

The image sensor module110is configured to capture a sequence of original image frames I1according to a first setting applied to the image sensor module110. The first setting comprises resolution, frame rate, exposure and/or others. The image sensor module110is also configured to transmit the sequence of original image frames I1to the image signal processor120. In one embodiment of the invention, the image sensor module110captures the sequence of original image frames I1in a first resolution, which can be the maximum resolution the image sensor module110capable to capture, and provides to the image signal processor120in a first frame rate, which can be the maximum frame rate the image sensor module110capable to provide.

The image signal processor120is configured to generate a sequence of preview image frames I2and a sequence of intermediate image frames I3, and transmit to the backend processing unit130. In one embodiment of the present invention, the image signal processor120is further configured to scale the sequence of original image frames I1to generate the sequence of preview image frames I2in a second resolution, which may be smaller than the first resolution of the original image frames I1so that the sequence of preview image frames I2may be displayed and/or further stored as thumbnails. The image signal processor120is also configured to scale the sequence of original image frames I1to generate the sequence of intermediate image frames I3in a third resolution, which may be equal to or smaller than the original first resolution. Typically, the second resolution of the preview image frames I2is set equal or smaller than the third resolution of the intermediate image frames I3, depending on system default setting or user setting. Compared with conventional image capture system, the image signal processor120of the invention is capable to provide both the sequence of preview image frames I2and the sequence of intermediate image frames I3in parallel with different resolutions and/or frame rates to the backend processing unit130. In one embodiment of the invention, the image signal processor120may provide the sequence of preview image frames I2of the second resolution to the preview module132in a second frame rate, and provide the sequence of intermediate image frames I3of the third resolution to the image application module134in a third frame rate, in which the second frame rate is higher than the third frame rate and the second resolution is lower than the third resolution.

The backend processing unit130includes a preview module132and an image application module134. The preview module132is configured to provide the sequence of preview image frames I2to the touch panel140for displaying. The image application module134is configured to buffer the sequence of intermediate image frames I3, extract a sequence of intermediate image frames I4out of the sequence of intermediate image frames I3, and encode the sequence of intermediate image frames I4so as to generate a sequence of encoded image frames I5to the memory unit150. The image application module134may extract the sequence of intermediate image frames I4in response to a user input received from, for example, the touch screen140, and encode them into the sequence of encoded image frames I5of designated format such as JPEG or MPEG, depending system default setting or user setting. The encoding may be performed by a suitable encoder implemented in either by hardware or software. The image application module134may also receive the user input via other input interface of the image capture system100, for example a hardware shutter key. The user input may be a command for capturing burst images, or a series of commands for capturing single image.

Please refer toFIG. 2, which illustrates a detailed diagram of the image application module134according to one embodiment of the present invention. As shown inFIG. 2, the image application module134includes a buffer136and an image encoder138. The buffer136is configured to buffer the sequence of intermediate image frames I3while the sequence of intermediate image frames I3are transmitted from the image signal processor120in a continuous manner. In one embodiment of the present invention, the buffer136is implemented as a ring buffer for continuously buffering the incoming sequence of intermediate image frames I3. In one embodiment of the invention, the buffer136is configured to store a predetermined number of intermediate image frames I3, and overwrite the oldest intermediate image frame with the newest intermediate frame continuously. The predetermined number of intermediate image frame I3may be set to buffer image frames covering a period of time or designated by user. In another embodiment of the invention, the buffer may be allocated specifically for the image capture system100or be requested when the image capture system100is enabled.

The buffer136is also configured to extract the sequence of intermediate image frames I4out of the sequence of intermediate image frames I3in response to an input command IC, where the input command IC determines a start and an end of the sequence of intermediate image frames I4. The input command IC may be received via the touch screen140or other input interface. The image encoder138is configured to encode the sequence of intermediate image frames I4to generate the sequence of encoded image frames I5, and is configured to transmit the sequence of sequence of encoded image frames I5to the memory unit150. The image encoder138may be a software encoder or a hardware encoder. A size of the buffer136and a processing capability of the image encoder138are primary factors of performance efficiency of the backend processing unit130. In one embodiment of the invention, the buffer136continuously receives new intermediate image frames I3and provides intermediate image frame I4to the image encoder138concurrently.

In one embodiment of the present invention, resolution of the sequence of intermediate image frames I3or the sequence of encoded image frames I5is assigned via another input command from the user and is generally higher than the resolution of the sequence of preview image frames I2. However, in another embodiment of the invention, the resolution of the preview image frame I2can be set the same as the intermediate image frames I3.

The touch screen140is configured to display the sequence of preview image frames I2and is configured to receive the input command IC from the user of the image capture system100. The input command IC can be a command for launching an image capture application utilizing the burst image capture method of the present invention and/or for performing various image capture operations.

In one embodiment of the present invention, the touch screen140may be split into a display unit for displaying the sequence of preview image frames I2and a touch panel for receiving the input command IC. In another embodiment of the invention, the input command IC may be received via other input mechanism, such as a hardware shutter key dedicated for image capture purpose.

The memory unit150is configured to store the sequence of encoded image frames I5. The memory unit150may be an eMMC memory, an SD card, a micro SD card and/or other suitable storage device connected internally or externally. In one embodiment of the present invention, the sequence of encoded image frames I5are stored in the memory unit150in JPEG files.

Generally speaking, the image capture system100of the invention, particularly the image signal processor120and the backend processing unit130, may process preview image frames and intermediate image frames (which may be encoded and stored as encoded image frames) in parallel by separate processing paths. The image signal processor120of the invention is capable to scale original image frames into preview image frames and intermediate image frames in different resolutions and provide them to the preview module132and the image application module134of the backend processing unit130in different frame rates respectively in parallel. Accordingly, the image capture system100of the invention may provide preview image frames on the touch screen140to the user, meanwhile receive input command IC for performing image capture (processed by the image application module134) at the same time. The input command IC may be a command for capturing a sequence of image frames, and in response to the input command IC, the buffer136provides the sequence of intermediate image frames I4corresponding to the input command IC to the image encoder for encoding into suitable format. Since the preview image frames I2and the intermediate image frames I3are processed by separate paths in parallel, the touch panel140can be updated with new image frames and the user is capable to capture burst images in high resolution nearly without delay.

Please refer toFIG. 3, which illustrates a burst image capture method according to a third embodiment of the present invention. As shown inFIG. 3, the burst image capture method comprises the following steps:

Step302: Capture a sequence of image frames I.

Step306: Provide the sequence of image frames I along a first path for preview and provide the sequence of image frames I into a buffer along a second path concurrently.

Step308: Receive an input command IC from a user via a user interface unit.

Step310: Extract a portion of the sequence of image frame I from the buffer and encode the portion of image frames along the second path in response to the input command IC.

Step312: Store the portion of encoded image frames.

In Step302, the image sensor module110captures the sequence of image frames I and transmits the sequence of image frames I to the image signal processor120continuously once the image capture application is triggered. The image sensor module110may captures the sequence of the image frame I in a predetermined resolution at a predetermined frame rate. And the image signal processor120may perform various processing on the sequence of image frames I prior to outputting to backend processing unit130.

In Step306, the image signal processor120provide the sequence of image frames I along two paths: the first path connecting to the preview module132and the second path connecting to the image application module134. By providing the sequence of image frames I in two separate paths, the image capture system100is capable to provide preview of image frames and perform image capture of the image frames at the same time. The preview module132may process the sequence of image frames I and provides the processed image frames to the touch screen for display, meanwhile the image application module134may buffer the sequence of image frames I into the buffer136concurrently.

In Steps308and310, in response to an input command received from the user interface unit, for example touch screen140ofFIG. 1, the image application module extracts a portion of image frames corresponding to the input command from the buffer136, and encodes the extracted portion of the image frames by the image encoder138. Please note that in Step308, the image signal processor120continues to provide newly captured image frames to the preview module132along the first path and to the buffer136along the second path so that both can be updated with new image frames. And the image frames captured and buffered to the buffer136during the valid time period of the input command IC would be encoded by the encoder138at the same time. The valid time period of input command may be determined by user, for example tap to release, or may be predetermined.

In Step312, the encoded image frames are stored in memory unit150of the image capture system100. The memory unit150may be internal or external storage device, such as SD/micro SD card, eMMC, USB-connected memory, etc.

Please refer toFIG. 4, which illustrates the image capture method according to a first embodiment of the present invention. As shown inFIG. 4, the image capture method comprises the following steps:

Step402: Capture a sequence of original image frames I1.

Step404: Generate a sequence of preview image frames I2and the sequence of intermediate image frames I3from the sequence of original image frames I1. The sequence of preview image frames I2and the sequence of intermediate image frames I3may be of different resolutions than the sequence of original image frames I1

Step406: Provide the sequence of preview image frames I2to a preview module.

Step407: Buffer the sequence of intermediate image frames I3to an image application module. Please note that Steps406and407are performed concurrently and the preview image frames I2and the intermediate image frames I3are provided and buffered in different frame rates.

Step408: Receive an input command IC for capturing at least an image of a scene. The input command may be received from a touch screen or from a hardware key.

Step410: Extract a sequence of intermediate image frames I4out of the sequence of intermediate image frames I3and encode the sequence of intermediate image frames I4to generate the sequence of encoded image frames I5. Please note that the sequence of intermediate image frames I4is a consecutive portion of the sequence of intermediate image frames I3

Step412: Store the sequence of encoded image frames I5into a memory unit.

Note that Steps402-412may be performed in a continuous manner so that sequences of image frames are consecutively captured, previewed, buffered, encoded, and stored in embodiments of the present invention as long as the image capture system100is activated or an image capture application installed on the image capture system100is triggered in response to the input command IC.

In Step402, the image sensor module110captures the sequence of original image frames I1and transmits the sequence of original image frames I1to the image signal processor120continuously once the image capture application is triggered. The image sensor module110may captures the sequence of the original image frame I1in a predetermined resolution at a predetermined frame rate.

In Step404, the image signal processor120generates the sequence of image frames I2and the sequence of intermediate image frames I3from the sequence of original image frames I1according to a first setting, which indicates the resolution of the sequence of preview image frames I2and the resolution of the sequence of intermediate image frames I3. Note that in one embodiment related to the steps shown inFIG. 4, the resolution of the sequence of preview image frames I2is lower than the resolution of the sequence of original image frames I1and the sequence of preview image frames I2may be optionally scaled-down. And the resolution of the sequence of intermediate image frames I3is higher than the resolution of the sequence of preview images I2, and is lower than or equal to the resolution of the sequence of original image frames I1. Also note that the first setting may be dynamically changed by the user of the image capture system100via an input setting command received from the touch screen140, for example, the input command IC or another input command. In some embodiments of the present invention, a frame rate of capturing the sequence of original image frames I1by the image sensor module110may also be set in the first setting via the input command IC or another input command, for example, 4 or 5 image frames per second. In addition, the frame rate of providing the preview image frames I2may be different than the frame rate of buffering the intermediate image frame I3. For example, the preview image frame rate may be set higher than the buffering frame rate.

In Step406, the preview module132directs the sequence of preview image frames I2to the touch screen140for displaying to the user. In Step407, the image application module134buffers the sequence of intermediate image frames I3into the buffer136. Please note that displaying of the preview image frames I2and buffering of the intermediate image frames I3are performed concurrently in parallel by the backend processing unit130.

In Step408, when the user of the image capture system.100sees the sequence of preview image frames I2on the touch screen140, for example, as a preview image frame stream including a consecutive plurality of preview image frames, and the user may trigger an input command IC for capturing one or more image frames, for example via the touch screen140. The touch screen140receives the input command IC by detecting finger touch on the touch screen140. An image capture operation is therefore triggered in response to the input command IC.

In Step410, after the image capture operation is triggered in response to the input command IC, the image capturing module134starts extracting the sequence of intermediate image frames I4out of the sequence of intermediate image frames I3from the buffer136, and the image encoder138encodes the sequence of intermediate image frames I4into the sequence of encoded image frames I5, for example, as an encoded image frame stream while the image capture operation is triggered in a continuous manner.

In Step412, the sequence of encoded image frames I5are stored in the memory unit150for future browsing or further processing.

Note that Steps408-412may be continuously performed in response to the input command IC when the input command IC is continuously triggered. Actually, the input command IC may be regarded as a combination of two input commands, a first one of which is configured to activate the image capture operation and a second one of which is configured to terminate the image capture operation, i.e. stop execution of the first input command. More specifically, in one embodiment of the present invention, the first input command is triggered by pressing a long tap on the touch screen140, and the second touch command is triggered by releasing the long tap from the touch screen140. In another embodiment, the first input command is triggered by pressing a first short tap on the touch screen140and releasing the first short tap from the touch screen140right away, and the second input command is triggered by pressing a second short tap on the touch screen140and releasing the second short tap from the touch panel140right away. Yet in another embodiment of the present invention, the input command IC is triggered and terminated by a single tap on the touch screen140.

As a result of the burst image capture method shown inFIG. 4, prior to the user of the image capture system100triggers the input command IC via the touch screen140like pressing a shutter of the image capture system100, the sequence of original image frames I1are continuously captured by the image sensor module110. The sequence of preview image frames I2are continuously displayed on the touch panel140, and the sequence of intermediate image frames I3are continuously buffered in the buffer136. Upon the user of the image capture system100triggers the input command IC corresponding to an image capture operation and until the user stops execution of the image capture operation, the buffer136continuously extracts the sequence of intermediate image frames I4out of the buffered sequence of intermediate image frames I3to the image encoder138, the image encoder138continuously encodes the extracted sequence of intermediate image frames I4to generate the sequence of encoded image frames I5. And the memory unit150continuously stores the sequence of encoded intermediate image frames I5. Please note that operations of the buffer136, the image encoder138and the memory unit150are executed in parallel so that processing of the image frames would not be delayed. For example, a first image frame is stored into the memory unit150and a second image frame is encoded by the image encoder138and a third image frame is buffered into the buffer136at the same time. The first image frame is captured prior to the second image frame, and the second image frame is captured prior to the third image frame.

In aspects of user experience, the user of the image capture system100can benefit from utilizing a fast burst image capture operation since the image sensor module110needs not switch resolution setting among preview mode and capture mode. Instead, the image signal processor120separates the image frames for preview and for image capture so that the image sensor module110is freed from switching its settings repeatedly in comparison to conventional image capture system of mobile electronic device. With the aid of the image capture system of the present invention, the lags occur between capturing two consequent image frames in the conventional mobile electronic device can be successfully prevented.

Please refer toFIG. 5, which illustrates how a user of the image capture system100can trigger a burst image capture operation based on the burst image capture method of the present invention. As shown inFIG. 5, a sequence of image frames of a person PS who is currently detected in a scene by the image sensor module110is displayed on the touch screen140.

As mentioned, in one embodiment of the present invention, the user is able to tap and hold on a shutter button SB displayed on the touch screen140to continuously capture a sequence of image frames IS1, IS2, IS3, IS4, . . . and etc. until the user releases tap from the shutter button SB to terminate the image capture, i.e. the user may use a long tap to activate the image capture operation.

As mentioned above, in another embodiment of the present invention, the user is able to give a first short tap on the shutter button SB and release the first short tap from the shutter button SB right away for continuously capturing multiple image frames until the user gives a second short tap on the shutter button SB to terminate the execution of the image capture. That is, the user may use the first short tap to activate the burst image capture operation of the image capture system.100and may use the second short tap to terminate the burst image capture operation.

In addition, the touch screen140may display thumbnails IST1, IST2, etc. of captured image frames. The thumbnails are generated from preview image frame corresponding to the captured image frames and can displayed with the preview image frames at the same time. When performing image capture operation, the touch screen140is updated with the thumbnail of the latest captured image frame of the sequence of image frame IS. When the image capture operation terminates, the touch screen140can display thumbnail of the last captured image frame without further update. In one embodiment of the invention, the touch screen140may concurrently update the display of preview image frames and the display of thumbnails concurrently during execution of the image capture operation. In another embodiment of the invention, the touch screen140may only update either the preview image frame or the thumbnail.

The present invention discloses a burst image capture method and an image capture system utilizing the burst image capture method for the purpose of overcoming the tradeoff between the image capturing time and the resolution occurring in the prior art. By separating the image frames for previewing and for buffering/encoding, a fast and burst image capture operation can be achieved, and even the “zero shutter-lag” performance may also be fulfilled with the aid of the disclosed burst image capture method of the disclosed image capture system of the present invention.