Image sensing method and image sensing system

An image sensing method, comprising: (a) controlling an image sensor to apply a first frame rate to sense a first frame during a first frame period; (b) determining exposure variation in the first frame period; and (c) during one of the first frame period and a second frame period following the first frame period, switching the image sensor to apply a second frame rate different from the first frame rate to sense at least one third frame, when the step (b) determines the exposure variation exceeds a threshold level. A related image sensing system is also disclosed. Via such image sensing method, the frame switch delay of a conventional frame switch can be reduced.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image sensing method and an image sensing system, and particularly relates to an image sensing method an image sensing system which can save time for switching a frame rate of an image sensor.

2. Description of the Prior Art

A conventional image sensor always needs to switch between different frame rates. For example, the image sensor applies a low frame rate to sense frames in a standby mode, and applies a high frame rate to sense frames in an active mode.

FIG. 1is a schematic diagram illustrating a conventional frame rate switch. Each frame time allows the image sensor to capture one frame, and the image sensor will not capture more than one frame in a single frame time. The length of time of each frame time could be variant depends on the frame rate of the image sensor.

As illustrated inFIG. 1, the image sensor initially applies a low frame rate (i.e. low FPS, low frame per second) to sense a first frame and a second frame respectively in the same frame time FT_a, that is, the first/second frame period consume the same length of time (i.e. the first frame time FT_a).

In one example, an exposure variation occurs at a time point TP_C during the first frame period and the exposure variation is determined at a time point TP_D later than the time point TP_C and during the second frame period. The exposure variation can mean the brightness variation. For example, if an object such as a living creator approaches an image sensor, the brightness sensed by the image sensor changes such that exposure variation occurs.

Therefore, the frame rate switch will be triggered at a time point TP_St, which is during the second frame period. Such delay from TP_D to TP_St may be caused by frame data read out, or frame data processing such as amplifying, or any other reasons. As a result that the frame rate of the image sensor is finally switched to a high frame rate (i.e. high FPS, high frame per second) at the time point TP_Sr after the second frame period is lapsed. Then the image sensor is able to capture the following frames in the second frame time FT_b. That is, the frame rate switch is delayed for almost a whole second frame period after the exposure variation happened.

Such delay causes some disadvantages. For example, the image sensor is applied to a monitor, and the image sensor applies a low frame rate while the monitor is in a standby mode. Also, the monitor is activated and the image sensor switches to a high frame rate when a exposure variation occurs, since exposure variation may mean some persons shows up in the scene captured by the image sensor or some events need to be noticed occur.

However, due to above-mentioned delay of the frame rate switch, the image sensor slowly switches to the high frame rate, such that some critical frames may not be clearly sensed in time.

SUMMARY OF THE INVENTION

Therefore, one objective of the present invention is to provide an image sensing method which can reduce the frame rate switch delay.

Another objective of the present invention is to provide an image sensing system which can reduce the frame rate switch delay.

One embodiment of the present invention discloses an image sensing method, comprising: (a) controlling an image sensor to apply a first frame rate to sense a first frame during a first frame period; (b) determining exposure variation in the first frame period; and (c) during one of the first frame period and a second frame period following the first frame period, switching the image sensor to apply a second frame rate different from the first frame rate to sense at least one third frame, when the step (b) determines the exposure variation exceeds a threshold level.

Another embodiment of the present invention discloses an image sensing system comprising an image sensor and a control circuit. The control circuit is configured to perform steps of: (a) controlling the image sensor to apply a first frame rate to sense a first frame during a first frame period; (b) determining exposure variation in the first frame period; and (c) during one of the first frame period and a second frame period following the first frame period, switching the image sensor to apply a second frame rate different from the first frame rate to sense at least one third frame, when the step (b) determines the exposure variation exceeds a threshold level.

In view of above-mentioned embodiments, the frame rate switch delay can be reduced. Also, the frame data amount of each frame sensed at a high frame rate can also be reduced.

DETAILED DESCRIPTION

In following descriptions, different embodiments are provided to explain the concept of the present invention. Please note, each component in following embodiments can be implemented by hardware (e.g. a circuit, a device and a system) or hardware with software (e.g. processor installed with at least one program). Besides, the component can be separated to more components, or integrated to fewer components. Additionally, the term “first”, “second” in following descriptions are only for define the same kind but different components, and do not mean any sequence or arrangement.

Further, following embodiments are applied to an example for switching an image sensor from a low frame rate to a high frame rate. However, the following embodiments can be applied to switch the image sensor from a high frame rate to a low frame rate as well.

FIG. 2is a block diagram illustrating an image sensing system according to one embodiment of the present invention, and the image sensing system could be implemented as an IC, a module or a device. As illustrated inFIG. 2, the image sensing system200comprises an image sensor201, a control circuit203and a reading circuit205. The image sensor201is configured to sense frames, the reading circuit205is configured to read the sensed frames from the image sensor201and the control circuit203is configured to control the image sensor201to be operated in specific condition such as a specific frame rate or execute an exposure operation of the image sensor201and to control the reading circuit205to readout the sensed frames with specific gain value and to control cell (s) of the image sensor201to be readout. Operations of the image sensing system200will be described for more details later.

In following descriptions, two terms “frame time” and “frame period” are used. The term “frame time” means the time that sensing and reading of a frame occupies, and the frame period means a time period in which a frame is supposed to be sensed. The frame time can be smaller or equals to the frame period. In some embodiments, if the frame time is smaller than the frame period, the image sensor does not sense a next frame until the frame period ends.

In the embodiments ofFIG. 3andFIG. 4, a frame time is equal to a frame time period.FIG. 3is a schematic diagram illustrating an image sensing method according to one embodiment of the present invention. The image sensor201applies a first frame rate (in this embodiment, a low frame rate and the frame time is FT_1) to sense a first frame during a first frame period (the frame period marked by solid lines). In the embodiment ofFIG. 3, the exposure variation occurs at the time point TP_C during the first frame period.

The image sensor201is supposed to use the first frame time FT_1to sense a second frame during the second frame period (the frame time FT_1marked by dotted lines), if no exposure variation occurs during the first frame period and the second frame period. However, if the exposure variation is identified (for example the exposure variation value is over a threshold level), the control circuit203controls the image sensor201to stop sensing the second frame and to switch from a low frame rate to a high frame rate and uses a second frame time FT_2to capture the next frame.

For more detail, if exposure variation happens and the value is larger than a threshold level at the time point TP_C and the control circuit203knows such exposure variation occurs at the time point TP_St during the second frame period, the frame rate switch will be triggered at the time point TP_St. Immediately after a reset operation Re, the image sensor201is switched to a high frame rate at the time point TP_Sr without waiting the lapsed of the second frame period. Therefore, the control circuit203stops the image sensor201to capture the second frame so as to switch the image sensor201to capture a third frame in the high frame rate.

Please note, the frame rate can also be switched before the end of the first frame period as long as the time point TP_St is located within the first frame period. In this embodiment, the first frame will not be successfully captured and the image sensor201will use the high frame rate to capture the second frame and so the second frame period will consume the second frame time FT_2which is less than the first frame time FT_1. Related embodiments will be described for more details later.

The exposure variation could be identified in different ways. For example, if the summation or average of at least some of the readout brightness value from the reading circuit205exceeds a specific value (in such case the specific value is the threshold level) then the exposure variation is identified, or if at least a specific number of the readout brightness value from the reading circuit205exceeds a specific value (in such case the specific number and the specific value are combined as the threshold level) then the exposure variation is identified.

Additionally, the above-mentioned reset operation Re means reset parameters for the image sensor201or reset parameters for components related with the image sensor201. For example, a timing generator for the image senor201recounts from 0, wherein the timing generator may help the image sensor201to determine the length of each frame period. Also, stopping the exposure operation of a next frame if the exposure operation already starts, and the sensing charges generated by the reading circuit for the next frame is cleaned. Besides, analog circuits and digital circuits related with the image sensor201are switched to settings of the high frame rate and operate accordingly and immediately. After that, a new exposure operation starts. In one embodiment, the exposure values needed for the low frame rate and the high frame rate are the same. In such case, the exposure operation of a next frame is not stopped. The analog circuits and digital circuits related with the image sensor can be switched to settings of the high frame rate and operate accordingly and immediately.

In one embodiment, the reset operation Re can be performed in a very short time nearly zero, or the reset operation Re can be ignored or removed due to the circuit design. In such case, the time point TP_St at which the frame rate switch is triggered and the time point TP_Sr at which the frame rate switch is performed are the same, as illustrated inFIG. 4.

Due to hardware constraints or other laminations, the speed for processing the frame data is limited. Therefore, embodiments for reducing frame data amount of each frame sensed at a high frame rate are also provided inFIG. 5andFIG. 6.

In one embodiment, the sensing array of the image sensor could be adjusted to decrease processing load in high frame rate operation. Please refer toFIG. 5, the control circuit203controls the image sensor201to apply a first sensing region SR_1to sense the first frame at a low frame rate and to apply a second sensing region SR_2smaller than the first sensing region SR_1at a high frame rate. Since the pixel density of the first sensing region SR_1and the pixel density of the second sensing region SR_2are the same, the image sensor201can generate fewer frame data of each frame while applying the high frame rate by such mechanism.

Additionally, in another embodiment down-sampling can be used to decrease processing load in high frame rate operation. Please refer toFIG. 6, the control circuit203controls the image sensor201to perform no down-sampling while sensing the first frame at a low frame rate and to perform down-sampling while sensing the third frame at a high frame rate. By this way, the frame data amount of each frame sensed by applying a high frame rate can be reduced as well.

Please refer toFIG. 2andFIG. 7. In one embodiment, the image sensor201senses the above-mentioned first frame to generate first sensing units SU (e.g. sensing charges), and the reading circuit205is configured to read the first sensing units SU. The image sensor201performs an exposure operation (e.g. the exposure operation EX inFIG. 7) to sense the first frame to accordingly generate first sensing units. After that, the reading circuit205can perform a read operation (e.g. the read operation RO inFIG. 7) reads the first sensing units SU from the image sensor201, such that the control circuit203can determine whether exposure variation occurs in the first frame period according to the sensing units. Please note, such operations can be performed to any other frame rather than limited to the first frame.

The exposure operation means the image sensor receives light to generate a frame. The exposure time can be controlled, for example, the active or non-active of the image sensor, the on/off of a light source or a shutter/an aperture of a camera in which the image sensor is provided.

In the embodiment ofFIG. 7the first time is smaller than the frame period. InFIG. 7, the image sensor201is supposed to respectively sense a first frame and a second frame during a first frame period (the frame time FT_1marked by solid lines) and a second frame period (the frame time FT_1marked by dotted lines) if no frame variation occurs in the first frame period and the second frame period. Additionally, the exposure variation in this embodiment occurs at the time point TP_C, which is in a time that the exposure operation EX is performed. Also, the frame rate switch is triggered at the time point TP_St, after the reading operation RO. After that, the frame rate switch is performed at the time point TP_Sr in the first period, after the reset operation Re.

As above-mentioned, if no reset operation Re is performed or the time of the reset operation Re is very short and could be ignored, the time point TP_St and the time point TP_Sr can be the same. Please note, in the embodiment ofFIG. 7, the exposure time of the exposure operation EX is smaller than the first frame period and the reading operation RO is also performed in the first frame period. Therefore, the embodiment illustrated inFIG. 7can be illustrated as: the first frame is completely sensed before the first frame period ends (i.e. the exposure operation EX and the reading operation are performed), and the frame rate switch is performed in the first frame time period and after the first frame is completely sensed.

In the embodiment ofFIG. 8, the frame time is smaller than the frame period.FIG. 8is a schematic diagram illustrating an image sensing method according to another embodiment of the present invention. In such embodiment, the image sensor201is supposed to use the first frame time FT_1to sense a first frame during a first frame period (solid lines) and to use the first frame time FT_1to sense a second frame during a second frame period (dotted lines) if no exposure variation occurs in the first frame period and the second frame period.

In the embodiment ofFIG. 8, the exposure variation occurs at a time point TP_C while a read operation RO for the first frame is performed in a first time period. Therefore, the control circuit203could not determine such exposure variation occurs until the second frame is sensed in a following second time period. Therefore, the frame rate is switched at the time point TP_Sr during or after the second frame period.

In the embodiment ofFIG. 8, the frame rate is switched after the exposure operation EX and the read operation RO in the second frame period are completed. Accordingly, the frame rate switch is triggered at the time point TP_St and performed at the time point TP_Sr before the first frame time FT_1of the second frame period lapsed. Briefly, in the embodiment ofFIG. 8, the exposure variation occurs in the first frame period, and the frame rate switch is performed before the second frame period ends. As above-mentioned, if the time of the reset operation Re is very short or the reset operation Re is not needed, the time point TP_St and the time point TP_Sr are the same.

The frame time and the frame period in the embodiment ofFIG. 9are also the same.FIG. 9is a schematic diagram illustrating an image sensing method according to another embodiment of the present invention. In this embodiment, the exposure operation in the first frame period is divided into the first exposure operation EX_1, the second exposure operation EX_2, the third exposure operation EX_3and the fourth exposure operation EX_4, but the embodiment ofFIG. 9is not limited to have four non-continuous exposure operations. The read operation RO is performed after all exposure operations are completed.

In the embodiment ofFIG. 9, the control circuit203can determine the exposure variation occurs as long as the exposure variation occurs before the read operation RO is performed. Accordingly, the frame rate switch can be performed in a second frame period following the first frame period, and the frame rate switch will not be delayed. The non-continuous operations extend the time in which the exposure variation can be determined. Therefore, the embodiment illustrated inFIG. 9is also helpful to reduce the delay of a frame rate switch. Briefly, in the embodiment ofFIG. 9, since a long exposure operation (e.g. the exposure operation EX inFIG. 8) is separated into a plurality of short exposure operations and distributed in a longer time period (e.g. the exposure operations EX_1-EX_4inFIG. 9), the time period in which the exposure variation can be detected is extended. Therefore, it is less likely to miss the exposure variation in the embodiment ofFIG. 9.

In view of above-mentioned embodiments, the frame rate switch delay can be reduced. Also, the frame data amount of each frame sensed at a high frame rate can also be reduced. However, it will be appreciated that the present invention is not limited to solve such problem.