Patent Description:
A solid-state imaging device, such as a charge coupled device (CCD) sensor and a complementary metal-oxide semiconductor (CMOS) sensor, is used in an image capturing apparatus, such as a digital camera and a digital video camera.

Generally, the dynamic range of the solid-state imaging device is known to be narrow compared with the dynamic range in the natural world. Thus, if a scene having a wide dynamic range (a high dynamic range) is imaged by a normal method, an underexposure, an overexposure, or the like may occur.

To address this, <CIT> proposes a technique in which a screen to be imaged is classified into a plurality of regions in accordance with the luminance and controls exposure conditions, such as a pixel exposure time and a gain of a pixel region by region. The technique discussed in <CIT> enables the imaging of the scene having the high dynamic range by determining the exposure conditions based on information acquired from preliminary imaging and performing main imaging in accordance with the exposure conditions.

<CIT> discusses a method, apparatus and computer program where signals are received from a camera unit having an image sensor with a plurality of image detectors. The signals include first signals received from first pixel detectors corresponding to a first image area and second signals received from second pixel detectors corresponding to a second image area surrounding the first image area. Camera settings of a setting group are automatically controlled based on the first signals. The settings group has at least one of focus control, exposure control and white balance control. Changes in the setting group are predicted based on the second signals. A change is detected in the first image area and used to adapt the automatic control of the camera settings of the settings group. Control signals are output to the camera unit to control the camera settings of the settings group.

<CIT> discusses providing dynamic adjustment to exposure metering used in digital image capture based upon a depth map. A computing device includes at least two image sensors that are synchronized to capture an image or frame of a scene at a same time. Some embodiments, prior to creating a digital image capture, generate a depth map based upon a current frame of a scene that is in view of the image sensors. In turn, the computing device generates weighting values based upon the depth map, and calculates a current frame luma based upon these weighting values. The computing device then calculates settings to adjust exposure metering based upon the current frame luma to improve the digital image capture relative to a digital image capture with fixed exposure metering.

<CIT> discusses methods and systems for predicting brightness of a scene. The predictions are used to set exposure settings of an imaging sensor, such as exposure time or sensor gain. In one aspect, a method include capturing an image with the imaging sensor, determining a pan direction of the imaging sensor, predicting a brightness of a next image based on the pan direction, adjusting a gain or an exposure time of the imaging sensor based on the brightness prediction, and capturing the next image using the adjusted image sensor gain or exposure time.

<CIT> discusses an aerial vehicle is configured to process an image captured by an imaging device, and to identify a portion of the image that is likely to appear in images subsequently captured by the imaging device based on the motion of the aerial vehicle. A control unit aboard the aerial vehicle generates instructions for controlling such motion and provides the instructions to the imaging device. Based on such instructions, the imaging device processes the image to identify a portion of the image that will appear within a field of view of the imaging device following the motion, and selects a shutter speed, an aperture, a level of gain, or another attribute of the imaging device based on the portion of the image, in order to optimize the quality of an image subsequently captured by the imaging device.

According to a first aspect of the present invention, there is provided an image capturing apparatus as specified in claims <NUM> to <NUM>. According to a second aspect of the present invention, there is provided a method for controlling an image capturing apparatus as specified in claim <NUM>. According to a third aspect of the present invention, there is provided a program as specified in claim <NUM>. According to a fourth aspect of the present invention, there is provided a non-transitory computer-readable storage medium as specified in claim <NUM>.

Further features of the present invention will become apparent from the following description of an example which does not form part of the invention but represents background art that is useful for understanding the invention, and embodiments with reference to the attached drawings.

In the following description, embodiments for implementing the present invention will be described in detail with reference to the accompanying drawings. The embodiments that will be described below are merely an example as how the present invention can be embodied, and shall be modified or changed as appropriate according to the configuration of an apparatus to which the present invention is applied and various kinds of conditions. The present invention is in no way limited to the following embodiments. Each of the embodiments of the present invention described below can be implemented solely or as a combination of a plurality of the embodiments or features thereof where necessary or where the combination of elements or features from individual embodiments in a single embodiment is beneficial.

An example will be described below. In the example, an image capturing apparatus corrects luminance distribution information about an image captured at the present imaging, based on a change between an angle of view used at the present imaging and an angle of view for the next imaging, thus acquiring luminance distribution information about an image to be captured for the next imaging. The image capturing apparatus determines exposure conditions for each region to be used for the next imaging based on the luminance distribution information about an image to be captured in the next imaging.

<FIG> illustrates an image capturing apparatus <NUM> according to the example and a network monitoring system including the image capturing apparatus <NUM>.

The image capturing apparatus <NUM> includes an image forming optical system <NUM>, a zoom control mechanism <NUM>, a solid-state imaging device unit <NUM>, a control unit <NUM>, and a transfer unit <NUM>.

The image forming optical system <NUM> includes a zoom lens, and the zoom control mechanism <NUM> can control a zoom function of the image forming optical system <NUM> and control the angle of view of the image forming optical system <NUM>.

Light introduced into the image forming optical system <NUM> is received by the solid-state imaging device unit <NUM>, and is imaged by being photoelectrically converted by the solid-state imaging device unit <NUM>. The solid-state imaging device unit <NUM> includes a large number of pixels, and the exposure condition of each of the pixels is individually controllable.

The image capturing apparatus <NUM> is capable of capturing a moving image, and the solid-state imaging device unit <NUM> generates captured image data corresponding to one frame of the moving image by carrying out imaging once.

The control unit <NUM> controls the solid-state imaging device unit <NUM>.

The transfer unit <NUM> is connected to an external client apparatus <NUM> via a wired or wireless network. The transfer unit <NUM> transfers image data processed by the control unit <NUM> to the client apparatus <NUM>. The external client apparatus <NUM> transmits a control command for, for example, controlling the angle of view of imaging of the image forming optical system <NUM> (i.e., zoom control) to the image capturing apparatus <NUM> via the network. The transfer unit <NUM> receives the control command. The image capturing apparatus <NUM> transmits a response to the control command to the client apparatus <NUM>.

The client apparatus <NUM> is, for example, a personal computer. The client apparatus <NUM> may supply electric power to the image capturing apparatus <NUM> via the network.

The control unit <NUM> is, for example a central processing unit (CPU), and operates in accordance with a computer program stored in a not-illustrated storage device.

The control unit <NUM> includes a region-by-region correction unit <NUM>, a development processing unit <NUM>, a region-by-region luminance information generation unit <NUM>, a movement information calculation unit <NUM>, and an exposure condition determination unit <NUM>. These elements are implemented by the control unit <NUM> executing the computer program. All of the elements in the control unit <NUM> are provided separately from the solid-state imaging device unit <NUM> in <FIG>, but a part or all of the functions of the control unit <NUM> may be included in the solid-state imaging device unit <NUM>.

The region-by-region correction unit <NUM> corrects brightness of each region in the captured image data provided by the solid-state imaging device unit <NUM>, and generates image data adjusted closer to brightness of each region in the real space.

The development processing unit <NUM> performs development processing, such as a white balance correction, Debayer, a noise reduction, a sharpness correction, and a gamma correction, on the image data provided by the region-by-region correction unit <NUM>. The image data processed by the development processing unit <NUM> is provided to the transfer unit <NUM>, and is transferred to the client apparatus <NUM>.

Alternatively, the captured image data or the image data may be transferred to the client apparatus <NUM> without being corrected by the region-by-region correction unit <NUM> and/or being subjected to the development processing by the development processing unit <NUM>.

The region-by-region luminance information generation unit <NUM> generates region-by-region luminance information based on the image data provided by the region-by-region correction unit <NUM>. The region-by-region luminance information is luminance distribution information indicating a luminance distribution in the image. The processing which is performed by the region-by-region luminance information generation unit <NUM> will be described with reference to <FIG>.

<FIG> illustrates the image of the image data captured by the solid-state imaging device unit <NUM> and corrected by the region-by-region correction unit <NUM>. As an example, <FIG> illustrates an image of an automobile that turns on the headlights at night, and the image has a high luminance contrast. <FIG> illustrates the luminance distribution indicated by the region-by-region luminance information generated by the region-by-region luminance information generation unit <NUM>. As it is clear from <FIG>, the region-by-region luminance information indicates such a luminance distribution that the vehicle body portion of the automobile is dark and the headlights and the region illuminated by the headlights are bright.

The region-by-region luminance information generation unit <NUM> provide the region-by-region luminance information to the exposure condition determination unit <NUM>.

<FIG> illustrates an example in which the image is divided into <NUM> regions horizontally and <NUM> regions vertically, but the number of regions in the image is not limited to this example and may be freely set in accordance with the number of pixels in the solid-state imaging device unit <NUM>. <FIG> illustrates an example of an image having five levels of luminance, but the number of levels of luminance is also not limited to this example and can be freely set.

Zoom control information (angle-of-view control information) for controlling the zoom control mechanism <NUM> is provided to the movement information calculation unit <NUM>. The zoom control information may be a control command provided by the transfer unit <NUM> or may be a control command that is provided by the transfer unit <NUM> and transferred from the zoom control mechanism <NUM>.

The movement information calculation unit <NUM> monitors the zoom control information, and determines whether there is a change between the angle of view used at the present imaging and the angle of view for the next imaging. If there is a change in the angle of view, the movement information calculation unit <NUM> calculates a distance information between respective positions, on the solid-state imaging device unit <NUM>, each corresponding to a different one of portions in the image captured at the present imaging and respective positions, on the solid-state imaging device unit <NUM>, each corresponding to a different one of the portions for the next imaging, based on the change in the angle of view. To put it another way, the movement information calculation unit <NUM> calculates, for a plurality of positions on the solid-state imaging device <NUM>, a movement amount between a first position on the solid-state imaging device <NUM>, corresponding to a portion in the captured image at the present imaging, and a second position on the solid-state imaging device <NUM>, corresponding to the same portion in the captured image at the next imaging. In other words, the movement information calculation unit <NUM> calculates a movement amount of the same portion between the present imaging frame and the next imaging frame. If there is a change in the angle of view, the movement information calculation unit <NUM> calculates a direction in which these positions move based on the change in the angle of view. A "position on the solid-state imaging device unit <NUM>" refers to a position contained in the imaging region of the solid-state imaging device unit <NUM>, and may be, for example, a pixel. The movement information calculation unit <NUM> calculates the movement amount and the movement direction of the position for a large number of portions in the image. The movement information calculation unit <NUM> provides movement information which is information indicating the calculated movement amount and movement direction to the exposure condition determination unit <NUM>.

<FIG> illustrates a change in the angle of view due to zoom control. An angle θ1 is an angle of view of imaging during wide-angle imaging (zoom-out), and θ2 is an angle of view of imaging during telephoto imaging (zoom-in). Assume that A represents the height of the entire imaging region of the solid-state imaging device unit <NUM>. The movement amount in the vertical direction between before and after the zoom control at a position on the solid-state imaging device unit <NUM> that corresponds to a certain portion in the image is calculated as a function of the position of this portion in the image captured at the present imaging, θ1, θ2, and A. The movement amount in the horizontal direction between before and after the zoom control at a position on the solid-state imaging device unit <NUM> that corresponds to a certain portion in the image is calculated as a function of the position of this portion in the image captured at the present imaging, θ1, θ2, and the width of the entire imaging region of the solid-state imaging device unit <NUM>. More specifically, the position corresponding to the central portion in the image is not changed between before and after the zoom control, but the position corresponding to another portion is changed according to the change in the angle of view between before and after the zoom control.

<FIG> illustrates the direction in which a position on the solid-state imaging device unit <NUM> moves during the zoom control.

Even in a case where the zoom-in control is performed, the position of the central portion in the field of view is not changed, but the peripheral portion is enlarged as indicated by the arrows at the next imaging. In the zoom-out control, the peripheral portion is reduced in size at the next imaging.

The movement information (the movement amounts and the directions of a large number of positions) calculated by the movement information calculation unit <NUM> and the region-by-region luminance information about the present imaging (the luminance distribution information about the image captured at the present imaging) that is generated by the region-by-region luminance information generation unit <NUM> are provided to the exposure condition determination unit <NUM>. The exposure condition determination unit <NUM> determines the exposure conditions for each region (e.g., for each pixel) to be used for the time of the next imaging based on the movement information calculated by the movement information calculation unit <NUM> and the region-by-region luminance information generated by the region-by-region luminance information generation unit <NUM>, and drives the solid-state imaging device unit <NUM> under the determined exposure conditions.

More specifically, the exposure condition determination unit <NUM> initially generates region-by-region luminance information (luminance distribution information) about the time of the next imaging by applying the movement information to the region-by-region luminance information about the present imaging.

In the case of the zoom-in control, the exposure condition determination unit <NUM> generates the region-by-region luminance information about the next imaging by applying the movement information schematically illustrated in <FIG> to the region-by-region luminance information about the present imaging that is exemplified in <FIG>. <FIG> illustrates the luminance distribution indicated by the region-by-region luminance information about the next imaging that is generated by the exposure condition determination unit <NUM>. The distribution in <FIG> corresponds to a distribution obtained by enlarging the distribution in <FIG> as a whole. More specifically, the distribution in <FIG> is enlarged twofold vertically and horizontally in <FIG> according to the change in the angle of view of imaging. Thus, <FIG> has the same brightness for every <NUM> × <NUM> pixels.

In the case of the zoom-out control, the exposure condition determination unit <NUM> generates the region-by-region luminance information about the next imaging so as to be able to acquire a distribution into which the distribution indicated by the region-by-region luminance information about the present imaging is reduced in size.

After generating the region-by-region luminance information about the next imaging, the exposure condition determination unit <NUM> determines the exposure conditions for each region to be used for the next imaging. More specifically, the exposure condition determination unit <NUM> determines an exposure value (an EV value) by applying the luminance of each region that is indicated by the region-by-region luminance information about the next imaging to a table for determining the exposure conditions exemplified in <FIG>. More specifically, the exposure condition determination unit <NUM> applies a low EV value to a region to which a bright luminance is specified by the region-by-region luminance information and applies a high EV value to a region to which a dark luminance is specified, thus preventing or reducing an overexposure and an underexposure. The exposure condition determination unit <NUM> notifies the solid-state imaging device unit <NUM> of the determined EV value. The solid-state imaging device unit <NUM> determines a charge accumulation time and a gain of each region based on the EV value that the solid-state imaging device unit <NUM> is notified of, and carries out the next imaging using the determined charge accumulation time and gain.

Alternatively, the exposure condition determination unit <NUM> may determine the charge accumulation time and the gain by applying the luminance of each region that is indicated by the region-by-region luminance information about the next imaging to the table for determining the exposure conditions exemplified in <FIG>. In this case, the solid-state imaging device unit <NUM> carries out the next imaging using the charge accumulation time and the gain that the solid-state imaging device unit <NUM> is notified of by the exposure condition determination unit <NUM>.

As described above, the region-by-region correction unit <NUM> corrects the brightness of each region for the captured image data provided by the solid-state imaging device unit <NUM>. In this correction, the region-by-region correction unit <NUM> uses the exposure conditions for each region that are determined by the exposure condition determination unit <NUM>, and acquires the brightness of each region in the real space without improvement in the exposure conditions for each region. This means that the region-by-region luminance information generated by the region-by-region luminance information generation unit <NUM> indicates a luminance distribution in the real space.

Conventional image capturing apparatuses determine the exposure conditions for the next imaging without consideration of a shift between an image captured at the present imaging and an image to be captured for the next imaging. Thus, the conventional image capturing apparatuses may expose a region corresponding to high luminance, such as the headlights of the automobile, with a high EV value, and expose a background region not illuminated by the headlights with a low EV value. Thus, an overexposure and/or an underexposure may undesirably occur due to the shift between the image captured at the present imaging and the image to be captured for the next imaging. However, the image capturing apparatus <NUM> according to the present example determines the exposure conditions for the next imaging in consideration of the shift between the image captured at the present imaging and the image to be captured for the next imaging. This configuration enables the image capturing apparatus <NUM> to perform imaging with prevention or reduction of an overexposure and an underexposure even in a case where the images are shifted between the present imaging and the next imaging according to the change in the angle of view due to the zoom control.

The operation of the image capturing apparatus <NUM> according to the example will be described with reference to a flowchart of <FIG>. The following description will focus on the flow of the processing for determining the exposure conditions for the next imaging, and will omit the detailed description of other processing.

First, in step S <NUM>, the image capturing apparatus <NUM> performs imaging (acquires captured image data corresponding to one frame of a moving image). Next in step S12, the region-by-region luminance information generation unit <NUM> generates the region-by-region luminance information. Next in step S13, the image capturing apparatus <NUM> determines whether the zoom control is performed for the next imaging. If the zoom control is performed (YES in step S13), the operation proceeds to step S14. If the zoom control is not performed (NO in step S13), the operation proceeds to step S16.

In step S14, the movement information calculation unit <NUM> calculates the movement information based on control information directed to the image capturing apparatus <NUM> (more specifically, the zoom control information), and provides the movement information to the exposure condition determination unit <NUM>. In step S15, the exposure condition determination unit <NUM> calculates the region-by-region luminance information about the next imaging by correcting the region-by-region luminance information about the present imaging that is provided by the region-by-region luminance information generation unit <NUM> using the movement information provided by the movement information calculation unit <NUM>. In step S <NUM>, the exposure condition determination unit <NUM> determines the exposure conditions for each region to be used for the next imaging based on the region-by-region luminance information to be used for the next imaging and the table for determining the exposure conditions. After step S16, the operation returns to step S11. In this manner, the solid-state imaging device unit <NUM> performs the imaging using the exposure conditions for each region that are determined by the exposure condition determination unit <NUM>.

If the determination in step S13 is NO (NO in step S13), the exposure condition determination unit <NUM> determines the exposure conditions for each region to be used for the next imaging by using, as the region-by-region luminance information about the next imaging, the region-by-region luminance information about the present imaging that is provided by the region-by-region luminance information generation unit <NUM>, without making a correction.

A first embodiment of the present invention will be described below. The image capturing apparatus according to the embodiment is configured in a similarly manner to the example described in conjunction with <FIG>. However, the image capturing apparatus <NUM> according to the first embodiment is different in terms of the method of generating the region-by-region luminance information (the luminance distribution information) about the image to be captured for the next imaging by the exposure condition determination unit <NUM>. In the present embodiment, the image capturing apparatus <NUM> calculates a luminance of a region of interest for the next imaging by performing interpolation using the luminance of the region of interest at the present imaging and luminances of a plurality of adj acent regions adj acent to the region of interest at the present imaging, in acquiring the luminance distribution information about the image to be captured for the next imaging based on the luminance distribution information about the image captured at the present imaging.

<FIG> illustrates the luminance distribution indicated by the region-by-region luminance information to be used for the next imaging that is generated by the exposure condition determination unit <NUM> for the zoom-in control according to the present embodiment. <FIG> corresponds to <FIG> according to the example. In other words, <FIG> corresponds to the region-by-region luminance information about the next imaging that the exposure condition determination unit <NUM> generates by applying the movement information schematically illustrated in <FIG> to the region-by-region luminance information about the present imaging that is exemplified in <FIG> in the case of the zoom-in control.

As seen from a comparison between <FIG> and <FIG>, the luminance of each of the regions is determined by performing interpolation using the luminance of this region and the luminances of regions surrounding it in <FIG>, and thus the same brightness is not necessarily shared for every <NUM> × <NUM> pixels as in <FIG>. In this manner, the image capturing apparatus <NUM> can perform imaging under further desirable exposure conditions by calculating the luminance of the region of interest for the time of the next imaging through interpolation using the luminance of the region of interest at the present imaging and the luminances of the plurality of adjacent regions adjacent to the region of interest at the present imaging in determining the region-by-region luminance information about the next imaging. In other words, an overexposure and an under exposure can be further prevented or reduced.

Here, the "region of interest" refers to a region contained in the imaging region of the solid-state imaging device unit <NUM>, and is formed by a plurality of pixels.

<FIG> is a schematic view in which the luminance distribution in the image before the zoom-in that is generated by the region-by-region luminance information generation unit <NUM> is enlarged into the luminance distribution in the image after the zoom-in without a change. In other words, <FIG> illustrates the luminance distribution indicated by the region-by-region luminance information generated by the exposure condition determination unit <NUM> according to the example. In such a case that a window letting in sunlight is present in an adjacent region <NUM> on the left side of a region <NUM>, and a dark wall is present in an adjacent region <NUM> on the right side of the region <NUM>, a boundary between the bright portion and the dark portion is located in the region <NUM>.

However, suppose that the luminance distribution illustrated in <FIG> is obtained actually after the zoom-in. In this case, a luminance of a region <NUM>, which is a left half of the region <NUM>, is lower than a luminance of the left-side adjacent region <NUM> and is higher than a luminance of a region <NUM>, which is a right half of the region <NUM>. The luminance of the region <NUM>, which is the right half of the region <NUM>, is lower than the luminance of the region <NUM>, which is the left half of the region <NUM>, and is higher than a luminance of the right-side adjacent region <NUM>. Thus, if imaging is performed in dependence upon the luminance distribution in <FIG>, the region <NUM>, which is the left half of the region <NUM>, is slightly underexposed while the region <NUM>, which is the right half of the region <NUM>, is slightly overexposed.

Moreover, as illustrated in <FIG>, hypothetically supposing that the luminance of the entire region <NUM> is determined to be an average value or an intermediate value of the luminance of the left-side adjacent region <NUM> and the luminance of the right-side adjacent region <NUM>, the region <NUM>, which is the left half of the region <NUM>, is slightly overexposed while the region <NUM>, which is the right half of the region <NUM>, is slightly underexposed.

<FIG> illustrates the luminance distribution for the next imaging that is generated by the exposure condition determination unit <NUM> according to the present embodiment. In the present embodiment, the exposure condition determination unit <NUM> calculates the luminance of the region of interest by performing interpolation using the luminance of the region of interest and the luminances of the plurality of adjacent regions adjacent to the region of interest. At this time, the relative positions of the regions <NUM> and <NUM> in the region <NUM> are also taken into consideration. Thus, the luminance of the region <NUM>, which is the left half of the region <NUM>, can be set to a relatively high value, and the luminance of the region <NUM>, which is the right half of the region <NUM>, can be set to and a relatively low value.

As a result, the region <NUM>, which is the left half, and the region <NUM>, which is the right half, each can be imaged under further appropriate exposure conditions.

Specifically, in a case where the zoom control mechanism <NUM> performs the zoom-in control between when the present imaging is performed and when the next imaging is performed, the exposure condition determination unit <NUM> can calculate a luminance K of the region of interest after the zoom-in (for the next imaging) using the following Equation <NUM>.

In Equation <NUM>, C, L, R, U, and D represent the luminances of the region of interest, the left-side adjacent region, the right-side adjacent region, the upper-side adjacent region, and the lower-side adjacent region before the zoom-in (at the present imaging), respectively, (refer to <FIG>). A variable α represents a relative distance from the center of gravity of the region of interest after the zoom-in (for the next imaging) to the left-side adjacent region before the zoom-in (at the present imaging), and β represents a relative distance from the center of gravity of the region of interest after the zoom-in (for the next imaging) to the upper-side adjacent region before the zoom-in (at the present imaging). Variables a and b are weighting coefficients. The weighting coefficient a determines the influence of the luminances L and R of the left and right adjacent regions at the present imaging on the luminance K of the region of interest for the next imaging. The luminances L and R of the left and right adjacent regions at the present imaging are more highly reflected in the luminance K of the region of interest for the next imaging as the weighting coefficient a increases, and the luminance C of the region of interest at the present imaging is more highly reflected in the luminance K of the region of interest for the next imaging as the weighting coefficient a reduces. The weighting coefficient b determines the influence of the luminances U and D of the upper and lower adjacent regions at the present imaging on the luminance K of the region of interest for the next imaging. The luminances U and D of the upper and lower adjacent regions at the present imaging are more highly reflected in the luminance K of the region of interest for the next imaging as the weighting coefficient b increases, and the luminance C of the region of interest at the present imaging is more highly reflected in the luminance K of the region of interest for the next imaging as the weighting coefficient b reduces.

Thus, the exposure condition determination unit <NUM> calculates the luminance of the region of interest for the next imaging based on the luminances of the region of interest and the adjacent regions at the present imaging, the relative distances from the center of gravity of the region of interest for the next imaging to the adjacent regions at the present imaging, and the movement amount and the direction calculated by the movement information calculation unit <NUM>.

The weighting coefficients a and b may be either the same as each other or different from each other. The exposure condition determination unit <NUM> may change the weighting coefficients a and b for each region even in the same image or may change the weighting coefficients a and b according to a scene. In the following description, an example of settings of the weighting coefficients a and b will be described.

It is desirable that the exposure condition determination unit <NUM> set the weighting coefficient a to a larger value as the difference between the luminance L of the left-side adjacent region and the luminance R of the right-side adjacent region at the present imaging increases. This is because it becomes more likely that the boundary between the bright and dark portions in the horizontal direction exists in the region of interest before the zoom-in as the difference between L and R increases. The exposure condition determination unit <NUM> calculates the luminance K of the region of interest for the next imaging in such a manner that the luminances L and R of these adjacent regions are highly reflected in the luminance K of the region of interest for the next imaging by setting the weighting coefficient a to a large value.

Similarly, it is desirable that the exposure condition determination unit <NUM> set the weighting coefficient b to a larger value as the difference between the luminance U of the upper-side adjacent region and the luminance D of the lower-side adjacent region at the present imaging increases. This is because it becomes more likely that the boundary between the bright and dark portions in the vertical direction exists in the region of interest before the zoom-in as the difference between U and D increases. The exposure condition determination unit <NUM> calculates the luminance K of the region of interest for the next imaging in such a manner that the luminances U and D of these adjacent regions are highly reflected in the luminance K of the region of interest for the next imaging by setting the weighting coefficient b to a large value.

In this manner, the exposure condition determination unit <NUM> calculates the luminance K of the region of interest for the next imaging in such a manner that the luminances of the adjacent regions are more highly reflected in the luminance K of the region of interest for the next imaging as the luminance difference increases between two adjacent regions that are adjacent to the region of interest at the present imaging and lined up in one direction(|L - R| or |U - D|).

In other words, as the difference reduces between the luminance L of the left-side adjacent region and the luminance R of the right-side adjacent region at the present imaging that are located in the opposite direction from each other, it becomes less likely that the boundary between the bright and dark portions in the horizontal direction exists in the region of interest before the zoom-in. Similarly, as the difference reduces between the luminance U of the upper-side adjacent region and the luminance D of the lower-side adjacent region at the present imaging, it becomes less likely that the boundary between the bright and dark portions in the vertical direction exists in the region of interest before the zoom-in. For this reason, it is desirable to reduce the value of a or b as the luminance difference between L and R or the luminance difference between U and D reduces. In particular, it is desirable to set a = b = <NUM> in a case where both the difference between L and R and the difference between U and D are smaller than a specific threshold value. In this case, the luminance C of the region of interest at the present imaging is supposed to be set as the luminance K of the region of interest for the next imaging without conducting the interpolation using the luminances of the adjacent regions (in Equation <NUM>, K is calculated to be K = C in the case of a = b = <NUM>).

In this case, the interpolation processing is supposed to be performed only in the region containing the boundary between the bright and dark portions, and thus, the load of the interpolation processing can be reduced.

In the case where the zoom control mechanism <NUM> performs the zoom-out control between the present imaging and the next imaging, the exposure condition determination unit <NUM> generates the region-by-region luminance information for the next imaging so as to be able to acquire a distribution into which the distribution indicated by the region-by-region luminance information about the present imaging is reduced.

In the case of the zoom-out control, a plurality of regions at the present imaging is supposed to be located in one region for the next imaging unlike in the case of the zoom-in control. Thus, it is desirable that the exposure condition determination unit <NUM> calculate the luminance of the region of interest for the next imaging by averaging the luminances of a plurality of regions at the present imaging (the luminance of the region of interest at the present imaging and the luminances of a plurality of adjacent regions adjacent to the region of interest at the present imaging).

In the case of the zoom-out control, the luminance information to be referred to is partially insufficient in the peripheral portion in the angle of view of imaging for the next imaging. Thus, it is desirable that the exposure condition determination unit <NUM> estimate the luminance of the region of interest located in the peripheral portion in the angle of view of imaging for the next imaging based on the luminances of the region of interest and adjacent regions at the present imaging.

Having described the embodiments of the present invention, the above descriptions are not intended to limit the present invention, and various exemplary modifications including omission, addition, and replacement of a component are possible within the technical scope of the present invention as long as they fall into the scope of the appended claims.

For example, in the above-described embodiments, the exposure conditions for each region are determined at the time of imaging of each frame in the moving image. Thus, the above-described embodiments do not involve a concept of preliminary imaging to acquire an image for reference that is not presented to a user and main imaging to acquire an image that is presented to the user as in the technique discussed in <CIT>. However, the above-described embodiments may be modified so as to determine the exposure conditions for each region in the main imaging to acquire an image that is presented to the user based on the preliminary imaging captured to acquire an image for reference that is not presented to the user.

In the above-described embodiments, the exposure conditions for each region to be used for the next imaging are determined in a case where the images are shifted between the present imaging and the next imaging according to the zoom control, i.e., a change in the angle of view. However, the exposure conditions for each region to be used for the next imaging may be determined in a case where the images are shifted between the present imaging and the next imaging according to a movement of the field of view of the camera accompanying a pan or a tilt instead of the zoom control. In this case, the movement information calculation unit <NUM> calculates the movement amount and the movement direction of the position for a large number of portions in the image based on control information about the pan or the tilt.

The present invention can also be embodied by processing that supplies a program capable of realizing one or more functions of the above-described embodiments to a system or an apparatus via a network or a recording medium, and causes one or more processors in a computer of this system or apparatus to read out and execute the program. In this case, the program (the program code) read out from the recording medium itself is supposed to realize the functions of the embodiments. The recording medium recording this program therein can constitute the present invention.

An operating system (OS) or the like running on the computer may partially or entirely perform the actual processing based on an instruction of the program read out by the computer, and the functions of the above-described embodiments may be realized by this processing.

At least a part of the functional blocks illustrated in <FIG> may be realized by hardware. In the case where the functional blocks illustrated in <FIG> are realized by hardware, this can be achieved by, for example, using a predetermined compiler to thereby automatically generate a dedicated circuit on a field-programmable gate array (FPGA) from a program for realizing each step. The functional blocks in <FIG> may also be realized as hardware by forming a gate array circuit in a manner similar to the FPGA. The functional blocks in <FIG> may also be realized by an application specific integrated circuit (ASIC).

The storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)?), a flash memory device, a memory card, and the like.

Claim 1:
An image capturing apparatus (<NUM>) comprising:
a solid-state imaging device unit (<NUM>) including a plurality of imaging regions for each of which exposure conditions are individually controllable;
a calculation means (<NUM>) configured to calculate, for a plurality of positions on the solid-state imaging device (<NUM>), a movement amount between a first position on the solid-state imaging device unit (<NUM>), corresponding to a portion in an image captured at a present imaging, and a second position on the solid-state imaging device unit (<NUM>), corresponding to the same portion in an image captured for a next imaging, and configured to calculate a direction in which the first and second positions move based on control information directed to the image capturing apparatus (<NUM>), wherein the control information includes zoom control information;
a determination means(<NUM>) configured to determine the exposure conditions for each of the plurality of imaging regions to be used for the next imaging based on luminance distribution information about a captured image at the present imaging, and the movement amount and the direction calculated by the calculation means (<NUM>); and
a change means (<NUM>) configured to change an angle of view by performing zoom-in control or zoom-out control between the present imaging and the next imaging,
wherein the calculation means (<NUM>) calculates the movement amount and the direction based on a change between an angle of view at the present imaging and an angle of view for the next imaging, and
characterized in that,
in a case where the change means (<NUM>) performs zoom-in control between the present imaging and the next imaging, the determination means (<NUM>) is configured to calculate a luminance of a region of interest for the next imaging by performing interpolation based on a luminance of the region of interest at the present imaging, luminances of a plurality of adj acent regions adjacent to the region of interest at the present imaging, a relative distance from a center of gravity of the region of interest for the next imaging to the adjacent region at the present imaging, and the movement amount and the direction calculated by the calculation means (<NUM>) to determine the exposure conditions for the region of interest for the next imaging.