Patent Description:
There is a known technique of setting a time interval of reactivation and an amount of change in focal point evaluation value with respect to a reference in reactivation, based on imaging conditions, camera statuses, and other conditions, when repeating focusing operation (<CIT>).

<CIT> discusses if an aperture of the objective-imaging is the same as an aperture when a state has turned into a focused state with autofocus before the objective-imaging or if an F value of the objective-imaging is equal to or greater than an F value when a state has turned into a focused state with autofocus before the objective-imaging, the focused state is maintained and the focusing of the objective-imaging is skipped. Even in other cases, an in-focus position is detected near the in-focus position before the objective-imaging. Alternatively, if the user selects a speed priority, the in-focus position is detected in a narrow range near the in-focus position before the objective-imaging. This enables to achieve both enough speed and focus accuracy of the main photographing.

<CIT> discusses an image capturing apparatus comprises an image sensor having focus detection pixels configured to detect a phase difference of the subject image, a calculation unit configured to calculate a defocus amount of the subject image based on signals of the focus detection pixels, a focus control unit configured to adjust focus of the imaging optical system based on the defocus amount that was calculated, and a determination unit configured to determine whether or not the imaging optical system is in an in-focus state by comparing the defocus amount to a predetermined threshold value, wherein the determination unit sets a larger value for the predetermined threshold value in a case where the imaging optical system focuses on a moving subject than in a case where the imaging optical system focuses on a still subject.

According to a first aspect of the present invention, there is provided a control apparatus as specified in claims <NUM> and <NUM>. According to a second aspect of the present invention, there is a method as specified in claim <NUM>. According to a third aspect of the present invention, there is provided a non-transitory computer-readable medium as specified in claim <NUM>.

Some exemplary embodiments will be described in detail below with reference to the drawings. The claimed invention relates to the third exemplary embodiment of <FIG>, <FIG>. The other embodiments are not according to the claimed invention and are present for illustration purposes only.

<FIG> is a diagram illustrating a functional configuration of a control apparatus (an imaging apparatus) according to a first exemplary embodiment.

Light that has passed through a lens group <NUM> (an imaging optical system) is incident on an image sensor <NUM>, via an optical filter <NUM>, a diaphragm <NUM>, and a color filter <NUM>.

The lens group <NUM> is an optical system that condenses incident light from an object on the image sensor <NUM>. The lens group <NUM> includes a focus lens for bringing an object into focus, and a zoom lens for adjusting the angle of view.

For example, an infrared cut filter (IRCF) is disposed as the optical filter <NUM>.

The diaphragm <NUM> adjusts the quantity of the light to be incident on the image sensor <NUM> through the lens group <NUM> and the optical filter <NUM>.

The color filter <NUM> is disposed at each pixel of the light receiving surface of the image sensor <NUM> in predetermined order.

The image sensor <NUM> outputs captured-image information about an imaging target in an analog signal. Here, the image sensor <NUM> performs imaging and frame output a predetermined number of times per second (at a frame rate) successively.

A video image formed in the image sensor <NUM> undergoes gain control in an automatic gain controller (AGC) <NUM>. Subsequently, an analog-to-digital (A/D) conversion unit <NUM> converts the analog imaging signal into digital imaging signals.

A video signal processing unit <NUM> performs predetermined processing on the digital imaging signals from the A/D conversion unit <NUM>, and outputs a luminance signal and a color signal for each pixel. Further, the video signal processing unit <NUM> generates a video for output and generates parameters for camera control.

Here, the parameters for camera control include a parameter for aperture control, a focal point evaluation value (an automatic focus (AF) evaluation value) as a frequency component value for focusing (focusing operation), and a parameter to be used in white balance control for adjusting tint. These parameters are generated for each frame.

A video signal output unit <NUM> outputs the video signal generated by the video signal processing unit <NUM> to the outside.

An exposure control unit <NUM> calculates the luminance information in an imaging screen, based on the luminance information output from the video signal processing unit <NUM>. Subsequently, the exposure control unit <NUM> controls the diaphragm <NUM> and the AGC <NUM> to adjust the captured image to appropriate brightness. The brightness is also adjustable according to the storage time of the image sensor <NUM> adjusted by the shutter speed.

The focusing operation in the present exemplary embodiment involves the calculation of a contrast value or a high frequency component intensity from the video signal generated by the video signal processing unit <NUM>, to use them as the focal point evaluation value. An optical control unit <NUM> controls the lens group <NUM> to be at a position (an in-focus position) for maximizing the focal point evaluation value, which is a so-called contrast AF system.

An external setting unit <NUM> is used for typical camera operations including focusing, brightness designation, and zooming factor designation.

A control setting unit <NUM> sets camera control commands transmitted from the external setting unit <NUM>, and makes settings for exposure control, lens control, and the like. In one or more embodiments, the above-described functions referred to by the reference numerals <NUM> to <NUM> are included in an apparatus connected to the imaging apparatus via a network.

A procedure of processing of the present exemplary embodiment will be described with reference to a flowchart illustrated in <FIG>.

Here, the processing in <FIG> is executed each time the focal point evaluation value is generated by the video signal processing unit <NUM>.

In step S201, the optical control unit <NUM> determines whether an AF execution instruction issued by a user at the external setting unit <NUM> is received, via the control setting unit <NUM>. If the instruction is received (YES in step S201), the processing proceeds to step S204. Otherwise (NO in step S201), the processing proceeds to step S202.

In step S202, the optical control unit <NUM> performs processing of determining whether to execute AF processing (i.e., AF execution determination processing). The details of this processing will be described below.

In step S203, the optical control unit <NUM> determines whether an AF execution flag for issuing an instruction to perform the AF execution is True, based on the result of the processing in step S202. If the AF execution flag is True (YES in step S203), the processing proceeds to step S204. Otherwise (NO in step S203), this processing ends.

In step S204, the optical control unit <NUM> performs focus processing by AF.

Here, the details of the processing in step S204 will be described with reference to a flowchart illustrated in <FIG>.

First, in step S301, the optical control unit <NUM> starts a focus search operation. In the focus search operation, while driving the focus lens of the lens group <NUM> in a predetermined direction (the closest direction or infinite distance direction), the optical control unit <NUM> acquires the focal point evaluation value in step S302 and the position of the focus lens (hereinafter, the focus position) in step S303.

In step S304, the optical control unit <NUM> determines whether the focal point evaluation value acquired in step S302 is greater than the focal point evaluation value acquired last time. If greater (YES in step S304), the processing proceeds to step S305. Otherwise (NO in step S304), the processing proceeds to step S306.

In step S305, the optical control unit <NUM> updates the peak value of the focal point evaluation value. Specifically, if the current focal point evaluation value is larger than the peak focal point evaluation value (a focal point evaluation value stored as the peak value), the current focal point evaluation value is set (stored) as the peak focal point evaluation value. Further, the current focus position is set (stored) as the peak focus position (the focus lens position corresponding to the peak value of the focal point evaluation value).

In step S306, the optical control unit <NUM> sets the lens driving direction in the focus search operation to the direction opposite to the current direction. In other words, if the optical control unit <NUM> drives the lens in the closest direction currently, the optical control unit <NUM> sets the lens driving direction to the infinite distance direction; otherwise, the optical control unit <NUM> sets the lens driving direction currently in the infinite value direction to the closest direction.

If the focal point evaluation value increases during the current focus search, the optical control unit <NUM> updates the peak value without changing the driving direction of the focus lens, based on the assumption that a focus position will be in the current driving direction. On the other hand, if the focal point evaluation value becomes smaller than the last focal point evaluation value, the optical control unit <NUM> determines that the focus lens has moved away from the focus position, and then drives the focus lens in the direction opposite to the current moving direction.

In step S307, the optical control unit <NUM> determines whether the focus lens has reciprocated a predetermined number of times or more in the same area (the focus position). If the focus lens has reciprocated the predetermined number of times or more (YES in step S307), the processing proceeds to step S308. Otherwise (NO in step S307), the processing returns to step S301 to continue the focus search. This determination is based on the assumption that the stay of the focus lens within the same driving range means that an in-focus position will be in the range.

In step S308, the optical control unit <NUM> drives the focus lens to the peak focus position set in step S305, and then stops the focus lens (focus stop).

In step S309, the optical control unit <NUM> stores the current focal point evaluation value as the focal point evaluation value at the focus stop, and this processing ends. This focal point evaluation value (AF evaluation value) is to be used in the AF execution determination processing in step S202 to be described below.

<FIG> illustrates the relationship between focal point evaluation values and focus positions. At the end of a series of steps in the processing in <FIG>, the focus lens stops at the position (an in-focus position <NUM>) of the peak focal point evaluation value (an evaluation value peak <NUM>).

Next, the processing in step S202 will be described with reference to a flowchart illustrated in <FIG>.

In step S501, the optical control unit <NUM> calculates an amount of change in the focal point evaluation value. The amount of change is an absolute value difference between the focal point evaluation value (reference value) at the focus stop stored in step S309 and the focal point evaluation value of the current frame.

In step S502, the optical control unit <NUM> determines whether the amount of change in the focal point evaluation value calculated in step S501 is greater than or equal to a threshold. If the amount of change is greater than or equal to the threshold (YES in step S502), the processing proceeds to step S503. Otherwise (NO in step S502), the processing proceeds to step S504. The threshold is a value for adjusting sensitivity to an object change. With the threshold set to a low value, a response to even a slight object change easily occurs, but a response to disturbance such as noise can also occur. On the other hand, with the threshold set to a high value, a response to disturbance such as noise does not easily occur, but a response to an object change can be weak. Thus, it is suitable to adjust the threshold beforehand based on actual imaging scenes.

The processing in step S503 and step S504 is processing of accumulating the results of the determination in step S502. In this processing, the optical control unit <NUM> holds the results of the determination for the latest predetermined number of past frames, using a buffer that stores the result of the determination for each of the past frames.

In step S503, the optical control unit <NUM> sets the result of the determination corresponding to the current frame to <NUM>.

In step S504, the optical control unit <NUM> sets the result of the determination corresponding to the current frame to <NUM>.

In step S505, the optical control unit <NUM> calculates the total value of the results of the determination for the latest predetermined number of past frames.

In step S506, the optical control unit <NUM> determines whether the total value calculated in step S505 is greater than or equal to a threshold (a first value/AF execution threshold). If the total value is greater than or equal to the threshold (YES in step S506), the processing proceeds to step S507. Otherwise (NO in step S506), the processing proceeds to step S509. The threshold (the first value) in step S506 is a parameter for controlling the responsiveness that is the time interval between the occurrence of the object change and the AF execution. With the threshold (the first value) set to a high value, a response to a temporary object change due to, for example, a person cutting across in front of the object is reduced, but the responsiveness is low accordingly. With the threshold set a low value, the responsiveness is increased, but a response to a temporary object change can also easily occur. Thus, it is suitable that the threshold (the first value) is set based on imaging scenes.

In step S505, the optical control unit <NUM> also sets the predetermined number of frames (a second value) to calculate the total value in association with the threshold (the first value) to satisfy the threshold ≤ the number of predetermined frames.

A specific setting example is as follows.

First, the optical control unit <NUM> determines the threshold (the first value/AF execution threshold) to achieve an appropriate responsiveness. For example, for the occurrence of a response to an object change in one second, the optical control unit <NUM> sets the threshold to <NUM> frames for an imaging apparatus with a frame rate of <NUM> fps. This is because a frame rate of <NUM> fps means that the time per frame is <NUM>/<NUM> seconds.

Next, the optical control unit <NUM> sets the predetermined number of frames to a value higher than or equal to the threshold. For example, with the predetermined number of frames set to <NUM>, which is the same as the threshold, the AF operation (focal point adjustment operation) is executed (started) only when the change in the focal point evaluation value of each of all the <NUM> frames is greater than or equal to the threshold of the focal point evaluation value in step S502.

The focal point evaluation value is sensitive to noise in an image or the like. That causes the AF operation to be executed fewer times in the environment where noise is easily generated at a high gain such as low illuminance imaging. This is because the focal point evaluation value based on an object change to which a response is supposed to occur becomes closer to focal point evaluation values based on noise, and reducing the amount of change in the total focal point evaluation value. Such a case therefore is less likely to occur with the predetermined number of frames set to a value larger than the threshold. In other words, the predetermined number of frames to be set depends on how much noise affecting the focal point evaluation value is acceptable. For example, the relationship between the predetermined number of frames and the threshold is determined based on amounts of noise in images.

In step S507, the optical control unit <NUM> clears (deletes) the buffer for storing the result of the determination.

In step S508, the optical control unit <NUM> sets the AF execution flag for issuing an instruction to execute (start) the AF operation to True, and this processing ends.

In step S509, the optical control unit <NUM> sets the AF execution flag to False, and this processing ends.

<FIG> exemplifies change in the focal point evaluation value in the present exemplary embodiment. <FIG> exemplifies the change in the total value calculated in step S505. In this example, the predetermined number of frames for calculating the total value are <NUM> frames (frames <NUM>), and the threshold (the first value/AF execution threshold) in step S506 is set to <NUM>. In other words, with the amounts of change in the focal point evaluation values of <NUM> frames out of the <NUM> frames larger than the threshold, the AF operation is executed. Further, a fixed frame rate (fps) means a fixed time per frame. With a fixed frame rate, the predetermined number of frames corresponds to a predetermined time. For example, if the frame rate is <NUM> fps, the time per frame is <NUM>/<NUM> seconds = <NUM> milliseconds. Hence, the above-described <NUM> frames out of the <NUM> frames means <NUM> milliseconds of the <NUM> milliseconds. The AF operation is therefore executed when the focal point evaluation value exceeds the threshold of the predetermined time (period).

A temporary object change due to, for example, a person cutting across in front of the object causes the amount of change in the focal point evaluation value to go high temporarily, as indicated by a section <NUM>. As illustrated in <FIG>, such a temporary object change however does not bring the total value to the threshold, which does not initiate the AF execution. On the other hand, continuous object change produces continuous high amounts of change in the focal point evaluation value, as indicated by a section <NUM>. In this case, the total value exceeds the threshold, initiating the AF execution.

As described above, according to the present exemplary embodiment, the AF execution is determined in consideration of the changes in the past focal point evaluation values, reducing a response to a temporary object change due to, for example, a person cutting across in front of the object. As a result, this configuration reduces blurs on video images or variations in angle of view due to an extra AF execution.

A second exemplary embodiment will be described. In the first exemplary embodiment, a method has been described of storing the results of the determination for the past frames for the predetermined number of frames, calculating the total value each time the results are stored, and determining the AF execution based on the calculated total value. In the second exemplary embodiment, the AF execution is determined in a simpler way without storing the determination results for the past frames or calculating the total value.

AF execution determination processing according to the present exemplary embodiment will be described with reference to a flowchart illustrated in <FIG>. The timing of starting the processing is similar to that in the first exemplary embodiment and thus the redundant description will not be omitted.

In step S601, the optical control unit <NUM> determines whether a predetermined number of frames have passed. The number of frames having passed is calculated based on the frame determined to have passed in step S601 last time. If the predetermined number of frames have passed (YES in step S601), the processing proceeds to step S602. Otherwise (NO in step S601), the processing proceeds to step S603. At the time of the initial execution, the result of the determination in step S601 is YES.

In step S602, the optical control unit <NUM> clears the count value. The count value will be described in detail below.

In step S603, the optical control unit <NUM> calculates the amount of change in the focal point evaluation value. The calculation method is similar to that in step S501 in the first exemplary embodiment.

In step S604, the optical control unit <NUM> determines whether the amount of change in the focal point evaluation value calculated in step S603 is greater than or equal to a threshold (a count threshold). The determination method is similar to that in step S502 in the first exemplary embodiment. If the optical control unit <NUM> determines that the amount of change is greater than or equal to the threshold (YES in step S604), the processing proceeds to step S605. Otherwise (NO in step S604), the processing proceeds to step S606.

In step S605, the optical control unit <NUM> increments the count value indicating the occurrence of a change in the focal point evaluation value, i.e., an object change.

In step S606, the optical control unit <NUM> determines whether the count value is greater than or equal to the threshold (the first value/AF execution threshold). If the count value is greater than or equal to the threshold (YES in step S606), the processing proceeds to step S607. Otherwise (NO in step S606), the processing proceeds to step S609.

In step S607, the optical control unit <NUM> clears the count value.

In step S608, the optical control unit <NUM> sets the AF execution flag for issuing an instruction to perform the AF execution to True, and this processing ends.

In step S609, the optical control unit <NUM> sets the AF execution flag for issuing an instruction to perform the AF execution to False, and this processing ends.

The predetermined number of frames in step S601 and the threshold in step S604 can each be set in a manner similar to that in the first exemplary embodiment.

<FIG> exemplifies change in the focal point evaluation value in the present exemplary embodiment. <FIG> exemplifies change in the count value calculated in step S605. In this example, the predetermined number of frames (the second value) for clearing (deleting) the count value are <NUM> frames (frames <NUM>), and the threshold (the first value) in step S606 is set to <NUM>. As with the first exemplary embodiment, a temporary object change due to, for example, a person cutting across in front of the object produces temporary high amounts of change in the focal point evaluation value, as indicated by a section <NUM>. In this case, as illustrated in <FIG>, the count value does not reach a threshold of <NUM> frames for clearing the count value, which does not initiate the AF execution. On the other hand, continuous object change produces continuous high amounts of change in the focal point evaluation value, as indicated by a section <NUM>. In this case, the count value in <FIG> exceeds the threshold, initiating the AF execution.

As described above, according to the present exemplary embodiment, without the operations of storing the determination result corresponding to each of the past frames and of calculating the total value, the AF execution is determined in a simple way as compared with the first exemplary embodiment.

A third exemplary embodiment will be described. In the second exemplary embodiment, a simple method has been described of incrementing the count value each time an object change is detected and performing the AF execution determination comparing the count value with the threshold. However, clearing the count value each time the predetermined number of frames have passed can cause a delay in the AF execution depending on the timing of an object change. For example, if the count value is cleared in response to the reach of the predetermined number of frames even though the count value is incremented due to detection of continuous object change, the AF execution can be delayed.

In the third exemplary embodiment, a method will be described of reducing this delay in simple processing as with the second exemplary embodiment.

<FIG> is a flowchart of the present exemplary embodiment. AF execution determination processing according to the present exemplary embodiment will be described with reference to the flowchart in <FIG>. Processing in steps similar to those in the above-described exemplary embodiments will not be described.

In step S702, the optical control unit <NUM> determines whether the number of consecutive counts to be described below is less than a predetermined value. If the number of consecutive counts is less than the predetermined value (YES in step S702), the processing proceeds to step S703. Otherwise (NO in step S702) the processing proceeds to step S705.

In step S704, the optical control unit <NUM> clears the number of consecutive counts.

In step S708, the optical control unit <NUM> increments the number of consecutive counts indicating that a change in the focal point evaluation value, i.e., object change, is continuously detected.

In step S709, the optical control unit <NUM> clears the number of consecutive counts.

In step S712, the optical control unit <NUM> clears the number of consecutive counts.

<FIG> exemplifies change of the focal point evaluation value in the present exemplary embodiment. <FIG> exemplifies the change of the count value calculated in step S707. In this example, the predetermined number of frames for clearing the count value (the second value) are <NUM> frames (frames <NUM>), and the threshold (the first value) in step S710 is set to <NUM>. Further, the predetermined value of the number of consecutive counts in step S702 is set to <NUM>. As with the first exemplary embodiment, as a temporary object change due to, for example, a person cutting across in front of the object occurs, the amount of change in the focal point evaluation value becomes high temporarily, as indicated by a section <NUM>. In this case, as illustrated in <FIG>, the count value does not reach the threshold, not initiating the AF execution. On the other hand, as continuous object change appears, the amount of change in the focal point evaluation values stays high continuously, as indicated by a section <NUM>. In this case, the count value in <FIG> exceeds the threshold, initiating the AF execution. Further, although the count value is cleared twice in the example described with reference to <FIG> of the second exemplary embodiment, the second clearing of the count value is not performed in this example. This is because the number of consecutive counts at that time exceeds <NUM>. As a result, the threshold is reached faster and the AF operation is thereby executed earlier than in the example of the second exemplary embodiment.

According to the above-described processing, with the number of continuous counts greater than or equal to the predetermined value, i.e., as continuous object change is detected, the count value is not cleared in step S703 (the count value is maintained). This reduces the delay in the AF execution of the second exemplary embodiment.

The present invention can also be implemented by processing for supplying a program for implementing one or more functions in the above-described exemplary embodiments to a system or apparatus via a network or a storage medium and causing one or more processors in a computer of the system or apparatus to read and execute the program. The present invention can also be implemented by a circuit that implements one or more functions (for example, an application specific integrated circuit (ASIC)).

<FIG> is a block diagram illustrating a hardware configuration example of a computer to execute the processing according to each of the above-described exemplary embodiments using a program.

A control apparatus <NUM> includes a central processing unit (CPU) <NUM>, a read only memory (ROM) <NUM>, a random access memory (RAM) <NUM>, an external memory <NUM>, a communication interface (I/F) <NUM>, and a system bus <NUM>.

The CPU <NUM> generally controls operations in the control apparatus <NUM>, and controls each component (the ROM <NUM>, the RAM <NUM>, the external memory <NUM>, and the communication I/F <NUM>) via the system bus <NUM>.

The ROM <NUM> is a nonvolatile memory to store control programs for the CPU <NUM> to execute processing. In one or more embodiments, these programs are stored in the external memory <NUM> or in a removable storage medium.

The RAM <NUM> functions as a main memory, a work area, or the like of the CPU <NUM>. In other words, the CPU <NUM> loads a program for executing the processing from the ROM <NUM> into the RAM <NUM>, and runs the loaded program, thereby implementing various kinds of functional operation.

The external memory <NUM> stores, for example, various types of data and various kinds of information for the CPU <NUM> to execute processing using programs. The external memory <NUM> also stores, for example, various types of data and various kinds of information obtained by the execution of processing by the CPU <NUM> using programs.

The communication I/F <NUM> is an interface for communicating with an external apparatus. The communication I/F <NUM> is, for example, a local area network (LAN) interface.

The system bus <NUM> connects the CPU <NUM>, the ROM <NUM>, the RAM <NUM>, the external memory <NUM>, and the communication I/F <NUM> communicably.

Claim 1:
A control apparatus to control focal point adjustment operation of an imaging optical system (<NUM>), the control apparatus comprising:
a calculation means (<NUM>) configured to calculate a focal point evaluation value for each of a plurality of images captured successively by the imaging optical system (<NUM>);
a determination means (<NUM>) configured to determine (S706) whether an amount of change in the focal point evaluation value calculated by the calculation means (<NUM>) with respect to a reference value is greater than or equal to a threshold;
a count value calculation means (<NUM>) for determining (S707), as a count value, the number of times the amount of change is determined to be greater than or equal to the threshold;
a means for determining (S708), as a number of consecutive counts, the number of consecutive times the amount of change is determined to be greater than or equal to the threshold;
a storage means configured to store the count value and the number of consecutive counts; and
a control means (<NUM>, <NUM>) configured to control the focal point adjustment operation of the imaging optical system (<NUM>) to be executed in a case where the count value is greater than or equal to a first value,
wherein:
when determining that a predetermined number of frames has passed and the number of consecutive counts is less than a predetermined number, the control means is configured to clear the count value and the number of consecutive counts,
when determining that the predetermined number of frames has passed while the number of consecutive counts is greater than or equal to the predetermined number, the control means is not configured to clear the count value and the number of consecutive counts, and
when determining that the count value is greater than or equal to the first value, the control means is configured to clear the count value and the number of consecutive counts.